Approximation by finite mixtures of continuous density functions that vanish at infinity

Abstract

Given sufficiently many components, it is often cited that finite mixture models can approximate any other probability density function (pdf) to an arbitrary degree of accuracy. Unfortunately, the nature of this approximation result is often left unclear. We prove that finite mixture models constructed from pdfs in ${\mathcal{C}}_0$ can be used to conduct approximation of various classes of approximands in a number of different modes. That is, we prove approximands in ${\mathcal{C}}_0$ can be uniformly approximated, approximands in ${\mathcal{C}}_b$ can be uniformly approximated on compact sets, and approximands in ${\mathcal{L}}_p$ can be approximated with respect to the ${\mathcal{L}}_p$, for $p\in \left[1,\mathrm{\infty }\right)$. Furthermore, we also prove that measurable functions can be approximated, almost everywhere.

Keywords:

• Approximation theory
• probability density functions
• finite mixture models
• Riemann summation
• uniform approximation

PUBLIC INTEREST STATEMENT

Finite mixture models are an expansive and expressive class of probability models that have been successfully applied in many situations where data follow a complex generative process that may be highly heterogeneous. It has long been known that finite mixture models, under sufficient regularity conditions, can approximate any probability density functions to arbitrary degrees of accuracy, and such results have been established under varying assumptive restrictions. Our work seeks to provide the weakest set of assumptions in order to establish approximation theoretic results over the widest class of probability density problems, possible. The result provides further evidence towards the success of mixture models in applications, and provides mathematical guarantees to practitioners who apply mixture models in their analytic problems.

1 Introduction

Let $x$ be an element in the Euclidean space, defined by ${\mathbb{R}}^n$ and the norm ${∥\cdot ∥}_2$, for some $n\in \mathbb{N}$. Let $f:{\mathbb{R}}^n\to \mathbb{R}$ be a function, such that $f\ge 0$, everywhere, and $\int f\text{d}\lambda =1$, where $\lambda$ is the Lebesgue measure. We say that $f$ is a probability density function (pdf) on the domain ${\mathbb{R}}^n$ (an expression that we will drop, from hereon in). Let $g:{\mathbb{R}}^n\to \mathbb{R}$ be another pdf, and for each $m\in \mathbb{N}$, define the functional class:

${\mathcal{M}}_m^g=\left\{h:h\left(x\right)=\sum _{i=1}^m{c}_i\frac{1}{{\sigma }_i^n}g\left(\frac{x-{\mu }_i}{{\sigma }_i}\right),{\mu }_i\in {\mathbb{R}}^n,{\sigma }_i\in {\mathbb{R}}_{+},c\in {\mathbb{S}}^{m-1},i\in \left[m\right]\right\},$

where $c^{\mathrm{T}}=(c_1,\dots ,c_m)$, ${\mathbb{R}}_{+}=\left(0,\mathrm{\infty }\right)$,

${\mathbb{S}}^{m-1}=\left\{c\in {\mathbb{R}}^m:\sum _{i=1}^m{c}_i=1\text{ and }{c}_i\ge 0,\mathrm{\forall }i\in \left[m\right]\right\}\text{,}$

$\left[m\right]=\left\{1,\dots ,m\right\}$, and $h_m^{\varphi }\in {\mathcal{M}}_m^{\varphi }$ is the matrix transposition operator. We say that any $h\in {\mathcal{M}}_m^g$ is a $m-\mathrm{c}\mathrm{o}\mathrm{m}\mathrm{p}\mathrm{o}\mathrm{n}\mathrm{e}\mathrm{n}\mathrm{t}$ location-scale finite mixture of the pdf $g$.

The study of pdfs in the class ${\mathcal{M}}_m^g$ is an evergreen area of applied and technical research, in statistics. We point the interested reader to the many comprehensive books on the topic, such as Everitt & Hand (1981), Titterington et al. (1985), McLachlan & Basford (1988), Lindsay (1995), McLachlan & Peel (2000), Fruwirth-Schnatter (2006), Schlattmann (2009), Mengersen et al. (2011), and Fruwirth-Schnatter et al. (2019).

Much of the popularity of finite mixture models stem from the folk theorem, which states that for any density $f$, there exists an $h\in {\mathcal{M}}_m^g$, for some sufficiently large number of components $m\in \mathbb{N}$, such that $h$ approximates $f$ arbitrarily closely, in some sense. Examples of this folk theorem come in statements such as “provided the number of component densities is not bounded above, certain forms of mixture can be used to provide arbitrarily close approximation to a given probability distribution”(Titterington et al., 1985, p. 50), “the [mixture] model forms can fit any distribution and significantly increase model fit” (Walker and Ben-Akiva, 2011, p. 173), and “a mixture model can approximate almost any distribution” (Yona, 2011, p. 500). Other statements conveying the same sentiment are reported in Nguyen & McLachlan (2019). There is a sense of vagary in the reported statements, and little is ever made clear regarding the technical nature of the folk theorem.

In order to proceed, we require the following definitions. We say that $f$ is compactly supported on $\mathbb{K}\subset {\mathbb{R}}^n$, if $\mathbb{K}$ is compact and if ${\mathbf{1}}_{{\mathbb{K}}^{\mathrm{C}}}f=0$, where ${\mathbf{1}}_{\mathbb{X}}$ is the indicator function that takes value 1 when $x\in \mathbb{X}$ and $0$, elsewhere, and ${\left(\cdot \right)}^{\mathrm{C}}$ is the set complement operator (i.e., ${\mathbb{X}}^{\mathrm{C}}={\mathbb{R}}^n\mathrm{\setminus }\mathbb{X}$). Here, $\mathbb{X}$ is a generic subset of ${\mathbb{R}}^n$. Furthermore, we say that $f\in {\mathcal{L}}_p\left(\mathbb{X}\right)$ for any $1\le p<\mathrm{\infty }$, if

${∥f∥}_{{\mathcal{L}}_p\left(\mathbb{X}\right)}={\left(\int {\left|{\mathbf{1}}_{\mathbb{X}}f\right|}^p\mathrm{d}\lambda \right)}^{1/p}<\mathrm{\infty },$

and for $p=\mathrm{\infty }$, if

${∥f∥}_{{\mathcal{L}}_{\mathrm{\infty }}\left(\mathbb{X}\right)}=\mathrm{i}\mathrm{n}\mathrm{f}\left\{a\ge 0:\lambda \left(\left\{x\in \mathbb{X}:\left|f(x)\right|>a\right\}\right)=0\right\}<\mathrm{\infty },$

where we call ${∥\cdot ∥}_{{\mathcal{L}}_p\left(\mathbb{X}\right)}$ the ${\mathcal{L}}_p-\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}$ on $\mathbb{X}$. When $\mathbb{X}={\mathbb{R}}^n$, we shall write ${∥\cdot ∥}_{{\mathcal{L}}_p\left({\mathbb{R}}^n\right)}={∥\cdot ∥}_{{\mathcal{L}}_p}$. In addition, we define the so-called Kullback-Leibler divergence, see Kullback & Leibler, (1951), between any two pdfs $f$ and $g$ on $\mathbb{X}$ as

$\mathrm{K}{\mathrm{L}}_{\mathbb{X}}\left(f,g\right)=\int {\mathbf{1}}_{\mathbb{X}}flog\left(\frac{f}{g}\right)\mathrm{d}\lambda .$

In Nguyen & McLachlan (2019), the approximation of pdfs $f$ by the class ${\mathcal{M}}_m^g$ was explored in a restrictive setting. Let $\left\{h_m^g\right\}$ be a sequence of functions that draw elements from the nested sequence of sets $\left\{{\mathcal{M}}_m^g\right\}$ (i.e., $h_1^g\in {\mathcal{M}}_1^g,h_2^g\in {\mathcal{M}}_2^g,\dots$). The following result of Zeevi & Meir (1997) was presented in Nguyen & McLachlan (2019), along with a collection of its implications, such as the results of from Li & Barron (1999) and Rakhlin et al. (2005).

Theorem 1 (Zeevi and Meir, 1997). If

$f\in \left\{f:{\mathbf{1}}_{\mathbb{K}}f\ge \beta ,\beta >0\right\}\cap {\mathcal{L}}_2\left(\mathbb{K}\right)$

and $g$ are pdfs and $\mathbb{K}$ is compact, then there exists a sequence $\left\{h_m^g\right\}$ such that

$\underset{m\to \mathrm{\infty }}{lim}{∥f-h_m^g∥}_{{\mathcal{L}}_2\left(\mathbb{K}\right)}=0and\underset{m\to \mathrm{\infty }}{lim}K{L}_{\mathbb{K}}\left(f,h_m^g\right)=0.$

Although powerful, this result is restrictive in the sense that it only permits approximation in the ${\mathcal{L}}_2$ norm on compact sets $\mathbb{K}$, and that the result only allows for approximation of functions $f$ that are strictly positive on $\mathbb{K}$. In general, other modes of approximation are desirable, in particular, approximation in ${\mathcal{L}}_p-\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}$ for $p=1$ or $p=\mathrm{\infty }$ are of interest, where the latter case is generally referred to as uniform approximation. Furthermore, the strict-positivity assumption, and the restriction on compact sets limits the scope of applicability of Theorem 1. An example of an interesting application of extensions beyond Theorem 1 is within the ${\mathcal{L}}_1-\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}$ approximation framework of Devroye & Lugosi (2000).

Let $g:{\mathbb{R}}^n\to \mathbb{R}$ again be a pdf. Then, for each $m\in \mathbb{N}$, we define

${\mathcal{N}}_m^g=\left\{h:h\left(x\right)=\sum _{i=1}^m{c}_i\frac{1}{{\sigma }_i^n}g\left(\frac{x-{\mu }_i}{{\sigma }_i}\right),{\mu }_i\in {\mathbb{R}}^n,{\sigma }_i\in {\mathbb{R}}_{+},c_i\in \mathbb{R},i\in \left[m\right]\right\},$

which we call the set of $m-\mathrm{c}\mathrm{o}\mathrm{m}\mathrm{p}\mathrm{o}\mathrm{n}\mathrm{e}\mathrm{n}\mathrm{t}$ location-scale linear combinations of the pdf $g$. In the past, results regarding approximations of pdfs $f$ via functions $\eta \in {\mathcal{N}}_m^g$ have been more forthcoming. For example, in the case of $g=\varphi$, where

(1) $\varphi \left(x\right)={\left(2\pi \right)}^{-n/2}exp\left(-{∥x∥}_2^2/2\right),$(1)

is the standard normal pdf. Denoting the class of continuous functions with support on ${\mathbb{R}}^n$ by $\mathcal{C}$. We have the result that for every pdf $f$, compact set $\mathbb{K}\subset {\mathbb{R}}^n$, and $ϵ>0$, there exists an $m\in \mathbb{N}$ and $h\in {\mathcal{N}}_m^{\varphi }$, such that ${∥f-h∥}_{{\mathcal{L}}_{\mathrm{\infty }}\left(\mathbb{K}\right)}<ϵ$ (Sandberg, 2001, Lem. 1). Furthermore, upon defining the set of continuous functions that vanish at infinity by

$\text{{C\_0} = left{ {f in C:forall epsilon \&gt 0,exists ,,{rm{a}},,{rm{compact}},,mathbb{K} subset {{mathbb R}^{rm n}}{rm{,}},,{rm{such}},,{rm{that}},,{{left| {rm f}right|}\_{{mathcal{L}\_infty }left({{mathbb{K}^{rm C}}} right)}} lt epsilon } right}{rm{,}}}$

we also have the result: for every pdf $f\in {\mathcal{C}}_0$ and $ϵ>0$, there exists an $m\in \mathbb{N}$ and $h\in {\mathcal{N}}_m^{\varphi }$, such that ${∥f-h∥}_{{\mathcal{L}}_{\mathrm{\infty }}}<ϵ$[32, Thm. 2]. Both of the results from Sandberg (2001) are simple implications of the famous Stone-Weierstrass theorem (cf. Stone, 1948 and De Branges, 1959).

To the best of our knowledge, the strongest available claim that is made regarding the folk theorem, within a probabilistic or statistical context, is that of DasGupta (2008, Thm. 33.2). Let $\left\{{\eta }_m^g\right\}$ be a sequence of functions that draw elements from the nested sequence of sets $\left\{{\mathcal{N}}_m^g\right\}$, in the same manner as $\left\{h_m^g\right\}$. We paraphrase the claim without loss of fidelity, as follows:

Claim 1. If $f,g\in \mathcal{C}$ are pdfs and $\mathbb{K}\subset {\mathbb{R}}^n$ is compact, then there exists a sequence $\left\{{\eta }_m^g\right\}$, such that

$\underset{m\to \mathrm{\infty }}{lim}{∥f-{\eta }_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}\left(\mathbb{K}\right)}=0.$

Unfortunately, the proof of Claim 1 is not provided within DasGupta (2008). The only reference of the result is to an undisclosed location in Cheney & Light (2000), which, upon investigation, can be inferred to be Theorem 5 of Cheney & Light (2000), Ch. 20. It is further notable that there is no proof provided for the theorem. Instead, it is stated that the proof is similar to that of Theorem 1 in Cheney & Light (2000, Ch. 24), which is a reproduction of the proof for Xu et al. (1993, Lem. 3.1).

There is a major problem in applying the proof technique of Xu et al. (1993, Lem. 3.1) in order to prove Claim 1. The proof of [Xu et al. (1993, Lem. 3.1)] critically depends upon the statement that “there is no loss of generality in assuming that $f\left(x\right)=0$ for $x\in {\mathbb{R}}^n\mathrm{\setminus }2\mathbb{K}$“. Here, for $a\in {\mathbb{R}}_{+}$, $a\mathbb{K}=\left\{x\in {\mathbb{R}}^n:x=ay,y\in \mathbb{K}\right\}$. The assumption is necessary in order to write any convolution with $f$ and an arbitrary continuous function as an integral over a compact domain, and then to use a Riemann sum to approximate such an integral. Subsequently, such a proof technique does not work outside the class of continuous functions that are compactly supported on $a\mathbb{K}$. Thus, one cannot verify Claim 1 from the materials of Xu & Light, (1993), Cheney & Light (2000), and DasGupta (2008), alone.

Some recent results in the spirit of Claim 1 have been obtained by Nestoridis & Stefanopoulos, (2007) and Nestoridis et al. (2011), using methods from the study of universal series (see, for example, in Nestoridis & Papadimitropoulos, 2005).

Let

$\mathcal{W}=\left\{f\in {\mathcal{C}}_0:\sum _{y\in {\mathbb{Z}}^n}\underset{x\in {\left[0,1\right]}^n}{sup}\left|f\left(x+y\right)\right|<\mathrm{\infty }\right\}$

denote the so-called Wiener’s algebra (see, e.g. Feichtinger, 1977)) and let

$\mathcal{V}=\left\{f\in {\mathcal{C}}_0:\mathrm{\forall }x\in {\mathbb{R}}^n,\left|f\left(x\right)\right|\le \beta {\left(1+{∥x∥}_2\right)}^{-n-\theta },\beta ,\theta \in {\mathbb{R}}_{+}\right\}$

be a class of functions with tails decaying at a faster rate than $o\left({∥x∥}_2^n\right)$. In Nestoridis et al. (2011), it is noted that $\mathcal{V}\subset \mathcal{W}$. Further, let

$\text{{C\_C} = left{ {f in C:exists ,,{rm{a}},{rm{compact}},{rm{set}},mathbb{K}{rm{,}},,{rm{such}},,{rm{that}},,{{bf{1}}\_{{mathbb{K}^}}}rm f = 0} right}{rm{,}}}$

denote the set of compactly supported continuous functions. The following theorem was proved in Nestoridis & Stefanopoulos (2007).

Theorem 2 (Nestoridis and Stefanopoulos, 2007, Thm. 3.2). If $g\in \mathcal{V}$, then the following statements hold.

(a) For any $f\in {\mathcal{C}}_c$, there exists a sequence $\left\{{\eta }_m^g\right\}$ (${\eta }_m^g\in {\mathcal{N}}_m^g$), such that

$\underset{m\to \mathrm{\infty }}{lim}{∥f-{\eta }_m^g∥}_{{\mathcal{L}}_1}+{∥f-{\eta }_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}}=0.$

(b) For any $f\in {\mathcal{C}}_0$, there exists a sequence $\left\{{\eta }_m^g\right\}$ (${\eta }_m^g\in {\mathcal{N}}_m^g$), such that

$\underset{m\to \mathrm{\infty }}{lim}{∥f-{\eta }_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}}=0.$

(c) For any $1\le p<\mathrm{\infty }$ and $f\in {\mathcal{L}}_p$, there exists a sequence $\left\{{\eta }_m^g\right\}$ (${\eta }_m^g\in {\mathcal{N}}_m^g$), such that

$\underset{m\to \mathrm{\infty }}{lim}{∥f-{\eta }_m^g∥}_{{\mathcal{L}}_p}=0.$

(d) For any measurable $f$, there exists a sequence $\left\{{\eta }_m^g\right\}$ (${\eta }_m^g\in {\mathcal{N}}_m^g$), such that

$\underset{m\to \mathrm{\infty }}{lim}{\eta }_m^g=f,almosteverywhere.$

(e) If $\nu$ is a $\sigma -finite$ Borel measure on ${\mathbb{R}}^n$, then for any $\nu -measurable$ $f$, there exists a sequence $\left\{{\eta }_m^g\right\}$ (${\eta }_m^g\in {\mathcal{N}}_m^g$), such that

$\underset{m\to \mathrm{\infty }}{lim}{\eta }_m^g=f,$

almost everywhere, with respect to $\nu$.

The result was then improved upon, in Nestoridis et al. (2011), whereupon the more general space $\mathcal{W}$ was taken as a replacement for $\mathcal{V}$, in Theorem 2. Denote the class of bounded continuous functions by ${\mathcal{C}}_b=\mathcal{C}\cap {\mathcal{L}}_{\mathrm{\infty }}$. The following theorem was proved in Nestoridis et al. (2011).

Theorem 3 (Nestoridis et al., 2011, Thm. 3.2). If $g\in \mathcal{W}$, then the following statements are true.

(a) The conclusion of Theorem 2(a) holds, with ${\mathcal{C}}_c$ replaced by ${\mathcal{C}}_0\cap {\mathcal{L}}_1$.

(b) The conclusions of Theorem 2(b)–(e) hold.

(c) For any $f\in {\mathcal{C}}_b$ and compact $\mathbb{K}\subset {\mathbb{R}}^n$, there exists a sequence $\left\{{\eta }_m^g\right\}$, such that

$\underset{m\to \mathrm{\infty }}{lim}{∥f-{\eta }_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}\left(\mathbb{K}\right)}=0.$

Utilizing the techniques from Nestoridis & Stefanopoulos, (2007), Bacharoglou, (2010) proved a similar set of results to Theorem 2, under the restriction that $f$ is a non-negative function with support $\mathbb{R}$, using $g=\varphi$ (i.e. $g$ has form (1), where $n=1$) and taking $\left\{h_m^{\varphi }\right\}$ as the approximating sequence, instead of $\left\{{\eta }_m^g\right\}$. That is, the following result is obtained.

Theorem 4 (Bacharoglou, 2010, Cor. 2.5). If $f:\mathbb{R}\to {\mathbb{R}}_{+}\cup \left\{0\right\}$, then the following statements are true.

(a) For any pdf $f\in {\mathcal{C}}_c$, there exists a sequence $\left\{h_m^{\varphi }\right\}$ ($h_m^{\varphi }\in {\mathcal{M}}_m^{\varphi }$), such that

$\underset{m\to \mathrm{\infty }}{lim}{∥f-h_m^{\varphi }∥}_{{\mathcal{L}}_1}+{∥f-h_m^{\varphi }∥}_{{\mathcal{L}}_{\mathrm{\infty }}}=0.$

(b) For any $f\in {\mathcal{C}}_0$, such that ${∥f∥}_{{\mathcal{L}}_1}\le 1$, there exists a sequence $\left\{h_m^{\varphi }\right\}$ ($h_m^{\varphi }\in {\mathcal{M}}_m^{\varphi }$), such that

$\underset{m\to \mathrm{\infty }}{lim}{∥f-h_m^{\varphi }∥}_{{\mathcal{L}}_{\mathrm{\infty }}}=0.$

(c) For any $1<p<\mathrm{\infty }$ and $f\in \mathcal{C}\cap {\mathcal{L}}_p$, such that ${∥f∥}_{{\mathcal{L}}_1}\le 1$, there exists a sequence $\left\{h_m^{\varphi }\right\}$ ($h_m^{\varphi }\in {\mathcal{M}}_m^{\varphi }$), such that

$\underset{m\to \mathrm{\infty }}{lim}{∥f-h_m^{\varphi }∥}_{{\mathcal{L}}_p}=0.$

(d) For any measurable $f$, there exists a sequence $\left\{h_m^{\varphi }\right\}$ ($h_m^{\varphi }\in {\mathcal{M}}_m^{\varphi }$), such that

$\underset{m\to \mathrm{\infty }}{lim}{h}_m^{\varphi }=f,almosteverywhere.$

(e) For any pdf $f\in \mathcal{C}$, there exists a sequence $\left\{h_m^{\varphi }\right\}$ ($h_m^{\varphi }\in {\mathcal{M}}_m^{\varphi }$), such that

$\underset{m\to \mathrm{\infty }}{lim}{∥f-h_m^{\varphi }∥}_{{\mathcal{L}}_1}=0.$

To the best of our knowledge, Theorem 4 is the most complete characterization of the approximating capabilities of the mixture of normal distributions. However, it is restrictive in two ways. First, it does not permit the characterization of approximation via the class ${\mathcal{M}}_m^g$ for any $g$ except the normal pdf $\varphi$. Although $\varphi$ is traditionally the most common choice for $g$ in practice, the modern mixture model literature has seen the use of many more exotic component pdfs, such as the student-t pdf and its skew and modified variants (see, e.g. Peel & McLachlan, 2000, Forbes & Wraith, 2013, and Lee & McLachlan, 2016). Thus, its use is somewhat limited in the modern context. Furthermore, modern applications tend to call for $n>1$, further restricting the impact of the result as a theoretical bulwark for finite mixture modeling in practice. A remark in Bacharoglou, (2010) states that the result can be generalized to the case where $g\in \mathcal{V}$ instead of $g=\varphi$. However, no suggestions were proposed, regarding the generalization of Theorem 4 to the case of $n>1$.

In this article, we prove a novel set of results that largely generalize Theorem 4. Using techniques inspired by Donahue et al. (1997) and Cheney & Light, (2000), we are able to obtain a set of results regarding the approximation capability of the class of $m-\mathrm{c}\mathrm{o}\mathrm{m}\mathrm{p}\mathrm{o}\mathrm{n}\mathrm{e}\mathrm{n}\mathrm{t}$ mixture models ${\mathcal{M}}_m^g$, when $g\in {\mathcal{C}}_0$ or $g\in \mathcal{V}$, and for any $n\in \mathbb{N}$. By definition of $\mathcal{V}$, the majority of our results extend beyond the proposed possible generalizations of Theorem 4.

The article proceeds as follows: Our main theorem is stated and its separate parts are proved in Section 2. Comments and discussion are provided in Section 3. Necessary technical lemmas and results are also included, for reference, in the Appendix.

2. Main result

The remainder of the article is devoted to proving the following theorem.

Theorem 5 (Main result). If we assume that $f$ and $g$ are pdfs and that $g\in {\mathcal{C}}_0$, then the following statements are true.

(a) For any $f\in {\mathcal{C}}_0$, there exists a sequence $\left\{h_m^g\right\}$ ($h_m^g\in {\mathcal{M}}_m^g$), such that

$\underset{m\to \mathrm{\infty }}{lim}{∥f-h_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}}=0.$

(b) For any $f\in {\mathcal{C}}_b$ and compact $\mathbb{K}\subset {\mathbb{R}}^n$, there exists a sequence $\left\{h_m^g\right\}$ ($h_m^g\in {\mathcal{M}}_m^g$), such that

$\underset{m\to \mathrm{\infty }}{lim}{∥f-h_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}\left(\mathbb{K}\right)}=0.$

(c) For any $1<p<\mathrm{\infty }$ and $f\in {\mathcal{L}}_p$, there exists a sequence $\left\{h_m^g\right\}$ ($h_m^g\in {\mathcal{M}}_m^g$), such that

$\underset{m\to \mathrm{\infty }}{lim}{∥f-h_m^g∥}_{{\mathcal{L}}_p}=0.$

(d) For any measurable $f$, there exists a sequence $\left\{h_m^g\right\}$ ($h_m^g\in {\mathcal{M}}_m^g$), such that

$\underset{m\to \mathrm{\infty }}{lim}{h}_m^g=f,almosteverywhere.$

(e) If $\nu$ is a $\sigma -\mathrm{f}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{t}\mathrm{e}$ Borel measure on ${\mathbb{R}}^n$, then for any $\nu -measurable$ $f$, there exists a sequence $\left\{h_m^g\right\}$ ($h_m^g\in {\mathcal{M}}_m^g$), such that

$\underset{m\to \mathrm{\infty }}{lim}{h}_m^g=f,$

almost everywhere, with respect to $\nu$.

If we assume instead that $g\in \mathcal{V}$, then the following statement is also true.

(f) For any $f\in \mathcal{C}$, there exists a sequence $\left\{h_m^g\right\}$ ($h_m^g\in {\mathcal{M}}_m^g$), such that

$\underset{m\to \mathrm{\infty }}{lim}{∥f-h_m^g∥}_{{\mathcal{L}}_1}=0.$

2.1. Technical preliminaries

Before we begin to prove the main theorem, we establish some technical results regarding our class of component densities ${\mathcal{C}}_0$. Let $f,g\in {\mathcal{L}}_1$ and denote the convolution of $f$ and $g$ by $f\star g=g\star f$. Further, we denote the sequence of dilates of $g$ by $\left\{g_k:g_k\left(x\right)=k^{n}g\left(kx\right),k\in \mathbb{N}\right\}.$ The following result is an alternative to Lemma 5 and Corollary 1. Here, we replace a boundedness assumption on the approximand, in the aforementioned theorem by a vanishing at infinity assumption, instead.

Lemma 1. Let $g$ be a pdf and $f\in {\mathcal{C}}_0$, such that ${∥f∥}_{{\mathcal{L}}_{\mathrm{\infty }}}>0$. Then,

$\underset{k\to \mathrm{\infty }}{lim}{∥g_k\star f-f∥}_{{\mathcal{L}}_{\mathrm{\infty }}}=0.$

Proof. It suffices to show that for any $ϵ>0$, there exists a $k\left(ϵ\right)\in \mathbb{N}$, such that ${∥g_k\star f-f∥}_{{\mathcal{L}}_{\mathrm{\infty }}}<ϵ$, for all $k\ge k\left(ϵ\right)$. By Lemma 6, $f\in {\mathcal{C}}_b$, and thus ${∥f∥}_{{\mathcal{L}}_{\mathrm{\infty }}}<\mathrm{\infty }$. By making the substitution $z=kx$, we obtain for each $k$

$\int g_k\left(x\right)\mathrm{d}\lambda =\int k^{n}g\left(kx\right)\mathrm{d}\lambda =\int g\left(z\right)\mathrm{d}\lambda =1.$

By Corollary 1, we obtain ${lim}_{k\to \mathrm{\infty }}\int {\mathbf{1}}_{\left\{x:{∥x∥}_2>\delta \right\}}{g}_k\mathrm{d}\lambda =0$ and thus we can choose a $k\left(ϵ\right)$, such that

$\int {\mathbf{1}}_{\left\{x:{∥x∥}_2>\delta \right\}}{g}_k\mathrm{d}\lambda <\frac{ϵ}{4{∥f∥}_{{\mathcal{L}}_{\mathrm{\infty }}}}.$

Since $g$ is a pdf, we have

$\left|\left(g_k\star f\right)(x)-f(x)\right|=\left|\int g_k\left(y\right)\left[f\left(x-y\right)-f\left(x\right)\right]\mathrm{d}\lambda \left(y\right)\right|\le \int g_k\left(y\right)\left|f\left(x-y\right)-f\left(x\right)\right|\mathrm{d}\lambda \left(y\right).$

By uniform continuity, for any $ϵ>0$, there exists a $\delta \left(ϵ\right)>0$ such that $\left|f\left(x-y\right)-f\left(x\right)\right|<\epsilon /2$, for any $x,y\in {\mathbb{R}}^n$, such that ${∥y∥}_2<\delta \left(ϵ\right)$ (Lemma 6). Thus, on the one hand, for any $\delta \left(ϵ\right)$, we can pick a $k\left(ϵ\right)$ such that

$\int {\mathbf{1}}_{\left\{y:{∥y∥}_2>\delta \left(ϵ\right)\right\}}{g}_k\left(y\right)\left|f\left(x-y\right)-f\left(x\right)\right|\mathrm{d}\lambda \left(y\right)$

$\le 2{∥f∥}_{{\mathcal{L}}_{\mathrm{\infty }}}\int {\mathbf{1}}_{\left\{y:{∥y∥}_2>\delta \left(ϵ\right)\right\}}{g}_k\mathrm{d}\lambda$

(2) $\le 2{∥f∥}_{{\mathcal{L}}_{\mathrm{\infty }}}\times \frac{ϵ}{4{∥f∥}_{{\mathcal{L}}_{\mathrm{\infty }}}}=\frac{ϵ}{2},$(2)

and on the other hand

$\int {\mathbf{1}}_{\left\{y:{∥y∥}_2\le \delta \left(ϵ\right)\right\}}{g}_k\left(y\right)\left|f\left(x-y\right)-f\left(x\right)\right|\mathrm{d}\lambda \left(y\right)$

$\le \frac{ϵ}{2}\int {\mathbf{1}}_{\left\{y:{∥y∥}_2\le \delta \left(ϵ\right)\right\}}{g}_k\mathrm{d}\lambda$

(3) $\le \frac{ϵ}{2}\times 1=\frac{ϵ}{2}.$(3)

The proof is completed by summing (2) and (3).□

Lemma 2. If $f\in {\mathcal{C}}_0$ is such that $f\ge 0$, and $ϵ>0$, then there exists a $h\in {\mathcal{C}}_c$, such that $0\le h\le f$, and

${∥f-h∥}_{{\mathcal{L}}_{\mathrm{\infty }}}<ϵ$

Proof. Since $f\in {\mathcal{C}}_0$, there exists a compact $\mathbb{K}\subset {\mathbb{R}}^n$ such that ${∥f∥}_{{\mathcal{L}}_{\mathrm{\infty }}\left({\mathbb{K}}^{\mathrm{C}}\right)}<ϵ/2$. By Lemma 7, there exists some $g\in {\mathcal{C}}_c$, such that $0\le g\le 1$ and ${\mathbf{1}}_{\mathbb{K}}g=1$. Let $h=gf$, which implies that $h\ge 0$ and $0\le h\le f$. Furthermore, notice that ${\mathbf{1}}_{\mathbb{K}}\left(f-h\right)=0$ and ${∥h∥}_{{\mathcal{L}}_{\mathrm{\infty }}}\le {∥f∥}_{{\mathcal{L}}_{\mathrm{\infty }}}$, by construction. The proof is completed by observing that

$f-h{ }_{{\mathcal{L}}_{\mathrm{\infty }}}=f-h{ }_{{\mathcal{L}}_{\mathrm{\infty }}\left({\mathbb{K}}^{\mathrm{C}}\right)}$

$\le {∥f∥}_{{\mathcal{L}}_{\mathrm{\infty }}\left({\mathbb{K}}^{\mathrm{C}}\right)}+{∥h∥}_{{\mathcal{L}}_{\mathrm{\infty }}\left({\mathbb{K}}^{\mathrm{C}}\right)}$

$\text{ le 2{left| fright|\_{{mathcal{L}\_infty }left({{mathbb{K}^c}} right)}} \&lt ,epsilon {rm{.}}}$

For any $\delta >0$, uniformly continuous function $f$, let

$w\left(f,\delta \right)=\underset{\left\{x,y\in {\mathbb{R}}^n:{∥x-y∥}_2\le \delta \right\}}{sup}\left|f\left(x\right)-f\left(y\right)\right|$

denote the modulus of continuity of $f$. Furthermore, define the diameter of a set $\mathbb{X}\subset {\mathbb{R}}^n$ by $\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{m}\left(\mathbb{X}\right)={sup}_{x,y\in \mathbb{X}}{∥x-y∥}_2$ and denote an open ball, centered at $x\in {\mathbb{R}}^n$ with radius $r>0$ by $\mathbb{B}\left(x,r\right)=\left\{y\in {\mathbb{R}}^n:{∥x-y∥}_2<r\right\}$.

Notice that the class ${\mathcal{M}}_m^g$ can be parameterized as

${\mathcal{M}}_m^g=\left\{h:h\left(x\right)=\sum _{i=1}^m{c}_i{k}_i^{n}g\left(k_{i}x-z_i\right),z_i\in {\mathbb{R}}^n,k_i\in {\mathbb{R}}_{+},c\in {\mathbb{S}}^{m-1},i\in \left[m\right]\right\},$

where $k_i=1/{\sigma }_i$ and $z_i={\mu }_i/{\sigma }_i$. The following result is the primary mechanism that permits us to construct finite mixture approximations for convolutions of form $g_k\star f$. The argument motivated by the approaches taken in Theorem 1 in Cheney & Light (2000, Ch. 24),  Nestoridis & Stefanopoulos (2007, Lem. 3.1), and Nestoridis et al. (2011, Thm. 3.1).

Lemma 3. Let $f\in \mathcal{C}$ and $g\in {\mathcal{C}}_0$ be pdfs. Furthermore, let $\mathbb{K}\subset {\mathbb{R}}^n$ be compact and $h\in {\mathcal{C}}_c$, where ${\mathbf{1}}_{{\mathbb{K}}^c}h=0$ and $0\le h\le f$. Then for any $k\in \mathbb{N}$, there exists a sequence $\left\{h_m^g\right\}$, such that

$\underset{m\to \mathrm{\infty }}{lim}{∥g_k\star h-h_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}}=0.$

Proof. It suffices to show that for any $k\in \mathbb{N}$ and $ϵ>0$, there exists a sufficiently large enough $m(ϵ)\in \mathbb{N}$ so that for all $m\text{gem}(ϵ),h_m^g\in {\mathcal{M}}_m^g$ such that

(4) ${∥g_k\star h-h_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}}<ϵ.$(4)

For any $k\in \mathbb{N}$, we can write

$\left(g_k\star h\right)(x)=\int g_k\left(x-y\right)h\left(y\right)\mathrm{d}\lambda \left(y\right)$

$=\int {\mathbf{1}}_{\left\{y:y\in \mathbb{K}\right\}}{g}_k\left(x-y\right)h\left(y\right)\mathrm{d}\lambda \left(y\right)$

$=\int {\mathbf{1}}_{\left\{y:y\in \mathbb{K}\right\}}{k}^{n}g\left(kx-ky\right)h\left(y\right)\mathrm{d}\lambda \left(y\right)$

$=\int {\mathbf{1}}_{\left\{z:z\in k\mathbb{K}\right\}}g\left(kx-z\right)h\left(\frac{z}{k}\right)\mathrm{d}\lambda \left(z\right).$

Here, $k\mathbb{K}$ is continuous image of a compact set, and hence is compact (cf. Rudin, 1976, Thm. 4.14]). By Lemma 8, for any $\delta >0$, there exists ${\kappa }_i\in {\mathbb{R}}^n$ ($i\in \left[m-1\right]$, $m\in \mathbb{N}$), such that $k\mathbb{K}\subset {\bigcup }_{i=1}^{m-1}\mathbb{B}\left({\kappa }_i,\delta /2\right)$. Further, if ${\mathbb{B}}_i^{\delta }=k\mathbb{K}\cap \mathbb{B}\left({\kappa }_i,\delta /2\right)$, then we have $k\mathbb{K}={\bigcup }_{i=1}^{m-1}{\mathbb{B}}_i^{\delta }$. We can obtain a disjoint covering of $k\mathbb{K}$ by taking ${\mathbb{A}}_1^{\delta }={\mathbb{B}}_1$ and ${\mathbb{A}}_i^{\delta }={\mathbb{B}}_i^{\delta }\mathrm{\setminus }{\bigcup }_{j=1}^{i-1}{\mathbb{B}}_j^{\delta }$ ($i\in \left[m-1\right]$) and noting that $k\mathbb{K}={\bigcup }_{i=1}^{m-1}{\mathbb{A}}_i^{\delta }$, by construction (cf. Cheney & Light, 2000, Ch. 24). Furthermore, each ${\mathbb{A}}_i^{\delta }$ is a Borel set and $\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{m}\left({\mathbb{A}}_i^{\delta }\right)\le \delta$.

For convenience, let ${\mathrm{\Pi }}_m^{\delta }=\left\{{\mathbb{A}}_i^{\delta }:i\in \left[m-1\right]\right\}$ denote the disjoint covering, or partition, of $k\mathbb{K}$. We seek to show that there exists an $m\in \mathbb{N}$ and ${\mathrm{\Pi }}_m^{\delta }$, such that

${∥g_k\star h-\sum _{i=1}^m{c}_i{k}_i^{n}g\left(k_{i}x-z_i\right)∥}_{{\mathcal{L}}_{\mathrm{\infty }}}<ϵ,$

where $k_i=k$,

$c_i=k^{-n}\int {\mathbf{1}}_{\left\{z:z\in {\mathbb{A}}_i^{\delta }\right\}}h\left(z/k\right)\mathrm{d}\lambda (z),$

and $z_i\in {\mathbb{A}}_i^{\delta }$, for $i\in \left[m-1\right]$.

Further, $z_m\in {\mathbb{A}}_{m-1}^{\delta }$ and $c_m=1-{\sum }_{i=1}^{m-1}{c}_i$, with $k_m$ chosen as follows: By Lemma 6, $g\le C<\mathrm{\infty }$ for some positive $C$. Then, ${∥c_m{k}_m^{n}g\left(k_{m}x-z_m\right)∥}_{{\mathcal{L}}_{\mathrm{\infty }}}\le c_m{k}_m^{n}C$. We may choose $k_m$ so that $k_m^n=ϵ/\left(2c_{m}C\right)$, so that

${∥c_m{k}_m^{n}g\left(k_{m}x-z_m\right)∥}_{{\mathcal{L}}_{\mathrm{\infty }}}\le \frac{ϵ}{2}.$

Since $0\le h\le f$, the sum of $c_i$ ($i\in \left[m-1\right]$) satisfies the inequality

$\sum _{i=1}^{m-1}{c}_i=k^{-n}\sum _{i=1}^{m-1}\int {\mathbf{1}}_{\left\{z:z\in {\mathbb{A}}_i^{\delta }\right\}}h\left(\frac{z}{k}\right)\mathrm{d}\lambda$

$=k^{-n}\int {\mathbf{1}}_{\left\{z:z\in k\mathbb{K}\right\}}h\left(\frac{z}{k}\right)\mathrm{d}\lambda$

$=\int {\mathbf{1}}_{\left\{x:x\in \mathbb{K}\right\}}h\mathrm{d}\lambda \le \int {\mathbf{1}}_{\left\{x:x\in \mathbb{K}\right\}}f\mathrm{d}\lambda \le \int f\mathrm{d}\lambda =1.$

Thus, $0\le c_m\le 1$, and our construction implies that $h_m^g\in {\mathcal{M}}_m^g,$ where

$h_m^g\left(x\right)=\sum _{i=1}^m{c}_i{k}_i^{n}g\left(k_{i}x-z_i\right)\mathrm{\forall }x\in {\mathbb{R}}^n.$

We can bound the left-hand side of (4) as follows:

${∥g_k\star h-h_g^m∥}_{{\mathcal{L}}_{\mathrm{\infty }}}$

$\le {∥\left(g_k\star h\right)\left(x\right)-\sum _{i=1}^{m-1}{c}_i{k}_i^{n}g\left(k_{i}x-z_i\right)∥}_{{\mathcal{L}}_{\mathrm{\infty }}}$

$+{∥c_m{k}_m^{n}g\left(k_{m}x-z_m\right)∥}_{{\mathcal{L}}_{\mathrm{\infty }}}$

$\le {∥\left(g_k\star h\right)\left(x\right)-\sum _{i=1}^{m-1}{c}_i{k}_i^{n}g\left(k_{i}x-z_i\right)∥}_{{\mathcal{L}}_{\mathrm{\infty }}}+\frac{ϵ}{2}$

$=\parallel \int {\mathbf{1}}_{\left\{z:z\in k\mathbb{K}\right\}}g\left(kx-z\right)h\left(\frac{z}{k}\right)\mathrm{d}\lambda \left(z\right)$

$-\sum _{i=1}^{m-1}\int {\mathbf{1}}_{\left\{z:z\in {\mathbb{A}}_i^{\delta }\right\}}g\left(kx-z_i\right)h\left(\frac{z}{k}\right)\mathrm{d}\lambda \left(z\right){\parallel }_{{\mathcal{L}}_{\mathrm{\infty }}}+\frac{ϵ}{2}$

(5) $\le \sum _{i=1}^{m-1}\int {\mathbf{1}}_{\left\{z:z\in {\mathbb{A}}_i^{\delta }\right\}}{∥g\left(kx-z\right)-g\left(kx-z_i\right)∥}_{{\mathcal{L}}_{\mathrm{\infty }}}h\left(\frac{z}{k}\right)\mathrm{d}\lambda \left(z\right)+\frac{ϵ}{2}.$(5)

Since

${∥kx-z-\left(kx-z_i\right)∥}_2={∥z-z_i∥}_2\le \mathrm{d}\mathrm{i}\mathrm{a}\mathrm{m}\left({\mathbb{A}}_i^{\delta }\right)\le \delta ,$

we have $\left|g\left(kx-z\right)-g\left(kx-z_i\right)\right|\le w\left(g,\delta \right)$, for each $i\in \left[m-1\right]$. Since ${lim}_{\delta \to 0}w\left(g,\delta \right)=0$ (cf. Makarov & Podkorytov, 2013, Thm. 4.7.3), we may choose a $\delta \left(ϵ\right)>0$ so that $w\left(g,\delta \left(ϵ\right)\right)<ϵ/\left(2k^n\right)$. We may proceed from (5) as follows:

${∥g_k\star h-h_g^m∥}_{{\mathcal{L}}_{\mathrm{\infty }}}\le w\left(g,\delta \left(ϵ\right)\right)\int {\mathbf{1}}_{\left\{z:z\in k\mathbb{K}\right\}}h\left(\frac{z}{k}\right)\mathrm{d}\lambda +\frac{ϵ}{2}$

$=w\left(g,\delta \left(ϵ\right)\right)k^n\int h\mathrm{d}\lambda +\frac{ϵ}{2}$

$\le w\left(g,\delta \left(ϵ\right)\right)k^n+\frac{ϵ}{2}$

(6) $<\frac{ϵ}{2}+\frac{ϵ}{2}=ϵ.$(6)

To conclude the proof, it suffices to choose an appropriate sequence of partitions ${\mathrm{\Pi }}_m^{\delta \left(ϵ\right)},m\text{gem}(ϵ)$, for some large but finite $m(ϵ)$, so that (5) and (6) hold, which is possible by Lemma 8.□

For any $r\in \mathbb{N}$, let ${\bar{\mathbb{B}}}_r=\left\{x\in {\mathbb{R}}^n:{∥x∥}_2\le r\right\}$ be a closed ball of radius $r$, centered at the origin.

Lemma 4. If $f\in {\mathcal{L}}_1$, such that $f\ge 0$, then

$\underset{r\to \mathrm{\infty }}{lim}{∥f-{\mathbf{1}}_{{\bar{\mathbb{B}}}_r}f∥}_{{\mathcal{L}}_1}=0.$

Proof. By construction, each element of the sequence $\left\{{\mathbf{1}}_{{\bar{\mathbb{B}}}_r}f\right\}$ ($r\in \mathbb{N}$) is measurable, $0\le {\mathbf{1}}_{{\bar{\mathbb{B}}}_r}f\le f$, and

$\underset{r\to \mathrm{\infty }}{lim}{\mathbf{1}}_{{\bar{\mathbb{B}}}_r}f=f,$

point-wise. We obtain our conclusion via the Lebesgue dominated convergence theorem.□

2.2. Proof of theorem 5(a)

We now proceed to prove each of the parts of Theorem 5. To prove Theorem 5(a) it suffices to show that for every $ϵ>0$, there exists a $h_m^g\in {\mathcal{M}}_m^g$, such that ${∥f-h_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}}<ϵ.$

Start by applying Lemma 2 to obtain $h\in {\mathcal{C}}_c$, such that $0\le h\le f$ and ${∥f-h∥}_{{\mathcal{L}}_{\mathrm{\infty }}}<ϵ/2$. Then, we have

${∥f-h_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}}\le {∥f-h∥}_{{\mathcal{L}}_{\mathrm{\infty }}}+{∥h-h_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}}$

(7) $<\frac{ϵ}{2}+{∥h-h_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}}.$(7)

The goal is to find a $h_m^g$, such that ${∥h-h_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}}<ϵ/2$. Since $h\in {\mathcal{C}}_c$, we may find a compact $\mathbb{K}\subset {\mathbb{R}}^n$ such that ${∥h∥}_{{\mathcal{L}}_{\mathrm{\infty }}\left({\mathbb{K}}^C\right)}=0$. Apply Lemma 1 to show the existence of a $k\left(ϵ\right)$, such that

${∥h-g_k\star h∥}_{{\mathcal{L}}_{\mathrm{\infty }}}<\frac{ϵ}{4},$

for all $k\ge k\left(ϵ\right)$. With a fixed $k=k\left(ϵ\right)$, apply Lemma 3 to show that there exists a $h_m^g\in {\mathcal{M}}_m^g$, such that

${∥g_{k\left(\epsilon \right)}\star h-h_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}}<\frac{ϵ}{4}.$

By the triangle inequality, we have

${∥h-h_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}}\le {∥h-g_{k\left(ϵ\right)}\star h∥}_{{\mathcal{L}}_{\mathrm{\infty }}}+{∥g_{k\left(ϵ\right)}\star h-h_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}}$

(8) $<\frac{ϵ}{4}+\frac{ϵ}{4}=\frac{ϵ}{2}.$(8)

The proof is complete by substitution of (8) into (7).

2.3. Proof of Theorem 5(b)

For any $ϵ>0$ and compact $\mathbb{K}\subset {\mathbb{R}}^n$, it suffices to show that there exists a sufficiently large enough $m(ϵ)\in \mathbb{N}$ so that for all $m\text{gem}(ϵ),h_m^g\in {\mathcal{M}}_m^g,$ such that ${∥f-h_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}(\mathbb{K})}<ϵ$.

By Lemma 5, we can find a $k\left(ϵ,\mathbb{K}\right)\in \mathbb{N}$, such that

(9) ${∥f-g_k\star f∥}_{{\mathcal{L}}_{\mathrm{\infty }}\left(\mathbb{K}\right)}<\frac{ϵ}{3},$(9)

for every $k\ge k\left(ϵ,\mathbb{K}\right)$. Since $g\in {\mathcal{C}}_0$, ${∥g∥}_{{\mathcal{L}}_{\mathrm{\infty }}}\le C<\mathrm{\infty }$ for some positive $C$, by Lemma 6. For any $k,r\in \mathbb{N}$, via Young’s convolution inequality:

(10) $\text{Vert g\_{k}star f-g\_{k}starleft(mathbf{1}\_{bar{mathbb{B}}\_r}fright)Vert \_{mathcal{L}\_{infty}} le k^{n}Cintleft(mathbf{1}\_{bar{mathbb{B}}\_r^{complement}}fright)text{d}lambda=k^{n}CVert f-{rm {1}}\_{Bar {mathbb{B}}\_r},{fVert \_{{mathcal L}\_{rm 1}}text{.}}$(10)

For fixed $k$, we may choose $r\left(ϵ,\mathbb{K}\right)\in \mathbb{N}$, using Lemma 4, so that ${∥f-{\mathbf{1}}_{{\bar{\mathbb{B}}}_r}f∥}_{{\mathcal{L}}_1}\le ϵ/\left(3k^{n}C\right)$ and thus the final term of (10) is bounded from above by $ϵ/3$ for all $r\ge r\left(ϵ,\mathbb{K}\right)$. Thus, for $k=k\left(ϵ,\mathbb{K}\right)$ and, $r\ge r\left(ϵ,\mathbb{K}\right)$

(11) ${∥g_{k\left(ϵ,\mathbb{K}\right)}\star f-g_{k\left(ϵ,\mathbb{K}\right)}\star \left({\mathbf{1}}_{{\bar{\mathbb{B}}}_{r\left(ϵ,\mathbb{K}\right)}}f\right)∥}_{{\mathcal{L}}_{\mathrm{\infty }}}\le \frac{ϵ}{3}.$(11)

Using Lemma 3, with approximand ${\mathbf{1}}_{{\bar{\mathbb{B}}}_{r\left(ϵ,\mathbb{K}\right)}}f$, component density $g$, compact set ${\bar{\mathbb{B}}}_{r\left(ϵ,\mathbb{K}\right)}$, $h={\mathbf{1}}_{{\bar{\mathbb{B}}}_{r\left(ϵ,\mathbb{K}\right)}}f$, and with $k=k\left(ϵ,\mathbb{K}\right)$ fixed, we have the existence of a density $h_m^g\in {\mathcal{M}}_m^g,m\text{gem}(ϵ)\in \mathbb{N},$ such that

(12) ${∥g_{k\left(ϵ,\mathbb{K}\right)}\star \left({\mathbf{1}}_{{\bar{\mathbb{B}}}_{r\left(ϵ,\mathbb{K}\right)}}f\right)-h_m^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}}\le \frac{ϵ}{3}.$(12)

We obtain the desired result by combining (9), (11), and (12), via the triangle inequality.

2.4. Proof of Theorem 5(c)

The technique used to prove Theorem 5(c) is different from those used in the previous sections. Here, we use a result of Donahue et al. (1997) that generalizes the classic Barron-Jones Hilbert space approximation result (cf. Jones, 1992 and Barron, 1993) to Banach spaces.

To prove Theorem 5(c), it suffices to show that for every $ϵ>0$, there exists a sufficiently large enough $m(ϵ)\in \mathbb{N}$ so that for all $m\text{gem}(ϵ),h_m^g\in {\mathcal{M}}_m^g$ such that ${∥f-h_m^g∥}_{{\mathcal{L}}_p}<ϵ$. Begin by applying Corollary 1 to obtain a $k\left(ϵ\right)$, such that

(13) ${∥f-g_k\star f∥}_{{\mathcal{L}}_p}<\frac{ϵ}{2}$(13)

for all $k\ge k\left(ϵ\right)$.

For some pdf $g$ and fixed $k\in \mathbb{N}$, let us define the class

${\mathcal{G}}_g^k=\left\{h:h\left(x\right)=k^{n}g\left(kx-k\mu \right),\mu \in {\mathbb{R}}^n\right\},$

write the $m-\mathrm{p}\mathrm{o}\mathrm{i}\mathrm{n}\mathrm{t}$ convex hull of ${\mathcal{G}}_g^k$ as

$\mathrm{C}\mathrm{o}\mathrm{n}{\mathrm{v}}_m\left({\mathcal{G}}_g^k\right)=\left\{h:h=\sum _{i=1}^m{c}_i{g}_i,g_i\in {\mathcal{G}}_g^k,c\in {\mathbb{S}}^{m-1},i\in \left[m\right]\right\},$

and call $\mathrm{C}\mathrm{o}\mathrm{n}{\mathrm{v}}_{\mathrm{\infty }}\left({\mathcal{G}}_g^k\right)=\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}\left({\mathcal{G}}_g^k\right)$ the convex hull of ${\mathcal{G}}_g^k$. We further say that $\overline{\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}\left({\mathcal{G}}_g^k\right)$ is the closure of $\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}\left({\mathcal{G}}_g^k\right)$.

Because $g$ is a pdf, $g\in {\mathcal{C}}_0\subset {\mathcal{C}}_b$, and ${\mathcal{C}}_b\subset {\mathcal{L}}_{\mathrm{\infty }}$, we observe that $g\in {\mathcal{L}}_1\cap {\mathcal{L}}_{\mathrm{\infty }}$. Thus, $g\in {\mathcal{L}}_p$, for any $1<p<\mathrm{\infty }$, by Lemma 9. Since $g$ is a pdf and $f\in {\mathcal{L}}_p$, we have the existence of $g_k\star f$ and the fact that ${∥g_k\star f∥}_{{\mathcal{L}}_p}$ is finite.

Furthermore, for any $\psi \in {\mathcal{G}}_g^k$, since $g\in {\mathcal{L}}_p$ and by definition of ${\mathcal{G}}_g^k$, we have ${∥\psi ∥}_{{\mathcal{L}}_p}\le k^{n/p}{∥g∥}_{{\mathcal{L}}_p}.$ Thus, we have

(14) ${∥\psi -g_k\star f∥}_{{\mathcal{L}}_p}\le {∥\psi ∥}_{{\mathcal{L}}_p}+{∥g_k\star f∥}_{{\mathcal{L}}_p}\le K,$(14)

by choosing $K=k^{n/p}{∥g∥}_{{\mathcal{L}}_p}+{∥g_k\star f∥}_{{\mathcal{L}}_p}>0$.

Following van de Geer (2003), we can write the closure of ${\mathcal{G}}_g^k$ as

$\overline{\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}\left(G_g^k\right)=\left\{h:h\left(x\right)=\int k^{n}g\left(kx-k\mu \right)f\left(\mu \right)\mathrm{d}\lambda \left(\mu \right),f\mathrm{i}\mathrm{s}\mathrm{a}\mathrm{p}\mathrm{d}\mathrm{f}\right\},$

and thus we immediately have $g_k\star f\in \overline{\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}\left(G_g^k\right)$. Combined with (14), we can apply Lemma 11 to obtain the conclusion that there exists a function $h_m^g\in \mathrm{C}\mathrm{o}\mathrm{n}{\mathrm{v}}_m\left({\mathcal{G}}_g^{k\left(ϵ\right)}\right)\subset {\mathcal{M}}_m^g$, such that

${∥h_m^g-g_{k\left(ϵ\right)}\star f∥}_{{\mathcal{L}}_p}\le \frac{K{C}_p}{m^{1-1/\alpha }},$

where $\alpha =min\left\{p,2\right\}$ and $C_p$ is a finite constant. Since $p>1$, $m^{1-1/\alpha }$ is strictly increasing, and hence we can choose an $m\left(ϵ\right)\in \mathbb{N}$, such that for all $m\text{gem}\left(ϵ\right)$,

(15) ${∥h_m^g-g_{k\left(ϵ\right)}\star f∥}_{{\mathcal{L}}_p}\le \frac{ϵ}{2}.$(15)

The proof is then completed by combining (13) and (15) via the triangle inequality.

2.5. Proof of Theorem 5(d) and Theorem 5(e)

By Theorem 5(a), there exists a sequence $\left\{h_m^g\right\}$ that uniformly converges to $f$, as $m\to \mathrm{\infty }$. Thus, by Lemma 12, $\left\{h_m^g\right\}$ almost uniformly converges to $f$ and also converges almost everywhere, to $f$, with respect to any measure $\nu$. We prove Theorem 5(d) by setting $\nu =\lambda ,$ and we prove Theorem 5(e) by not specifying $\nu$.

2.6. Proof of Theorem 5(f)

It suffices to show that for any $ϵ>0$, there exists a sufficiently large enough $m(ϵ)\in \mathbb{N}$ so that for all $m\text{gem}(ϵ),h_m^g\in {\mathcal{M}}_m^g$, where $g\in \mathcal{V}$, such that ${∥f-h_m^g∥}_{{\mathcal{L}}_1}<ϵ$. Begin by applying Lemma 4 in order to find a $r\left(ϵ\right)\in \mathbb{N}$, for any $ϵ>0$, such that for all $r\ge r\left(ϵ\right)$,

(16) ${∥f-{\mathbf{1}}_{{\bar{\mathbb{B}}}_r}f∥}_{{\mathcal{L}}_1}\le \frac{ϵ}{24}<\frac{ϵ}{2},$(16)

where $0\le {\mathbf{1}}_{{\bar{\mathbb{B}}}_r}f\le f$, and ${\mathbf{1}}_{{\bar{\mathbb{B}}}_r}f\in {\mathcal{C}}_c$ with compact support ${\bar{\mathbb{B}}}_r$.

Let $\mathbb{K}={\bar{\mathbb{B}}}_r$ and apply the triangle inequality to obtain

${∥f-h_m^g∥}_{{\mathcal{L}}_1}\le {∥f-{\mathbf{1}}_{\mathbb{K}}f∥}_{{\mathcal{L}}_1}+{∥{\mathbf{1}}_{\mathbb{K}}f-h_m^g∥}_{{\mathcal{L}}_1}$

$\le \frac{ϵ}{2}+{∥{\mathbf{1}}_{\mathbb{K}}f-h_m^g∥}_{{\mathcal{L}}_1}.$

Hence, we need to show that there exists a function $h_m^g\in {\mathcal{M}}_m^g$, such that

${∥{\mathbf{1}}_{\mathbb{K}}f-h_m^g∥}_{{\mathcal{L}}_1}\le \frac{ϵ}{2}.$

Since $g\in \mathcal{V}$ and $g_k(x)=k^{n}g(kx)$, by substitution, we have

(17) $g_k\left(x\right)\le \frac{\beta k^{-\theta }}{{\left(k^{-1}+{∥x∥}_2\right)}^{n+\theta }},$(17)

where $\beta ,\theta >0$ are independent of $k$. By Lemma 5 and Corollary 1, we can obtain a $k_1\left(ϵ\right)$, such that for all $k\ge k_1\left(ϵ\right)$,

(18) ${∥{\mathbf{1}}_{\mathbb{K}}f-g_k\star \left({\mathbf{1}}_{\mathbb{K}}f\right)∥}_{{\mathcal{L}}_1}\le \frac{ϵ}{4}.$(18)

Suppose that $\gamma >1$ and let

${\mathbb{K}}_k=\left\{x\in {\mathbb{R}}^n:\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{t}\left(x,\mathbb{K}\right)\le k^{-\gamma }\right\},$

where

$\mathrm{d}\mathrm{i}\mathrm{s}\mathrm{t}\left(x,\mathbb{X}\right)=\mathrm{i}\mathrm{n}\mathrm{f}\left\{{∥x-y∥}_2:y\in \mathbb{X}\right\}.$

By construction, $\lambda \left({\mathbb{K}}_k\right)=\lambda \left(\mathbb{K}\right)+O\left(k^{-\gamma }\right)$ and thus there exists a $k_2$ such that $\lambda \left({\mathbb{K}}_k\right)\le \lambda \left(\mathbb{K}\right)+1$, for any $k\ge k_2$.

For any $k>k_2$, we can show that

(19) ${∥g_k\star \left({\mathbf{1}}_{\mathbb{K}}f\right)-h_{m-1}^g∥}_{{\mathcal{L}}_1\left({\mathbb{K}}_k\right)}<\frac{ϵ}{8}.$(19)

To do so, firstly, for any $x\in {\mathbb{R}}^n$,

$g_k\star \left({\mathbf{1}}_{\mathbb{K}}f\right)=\int {\mathbf{1}}_{\mathbb{K}}{g}_k\left(x-y\right)f\left(y\right)\mathrm{d}\lambda \left(y\right)$

$=\int {\mathbf{1}}_{k\mathbb{K}}g\left(kx-z\right)f\left(\frac{z}{k}\right)\mathrm{d}\lambda \left(z\right).$

To obtain a Riemann sum approximation of $g_k\star \left({\mathbf{1}}_{\mathbb{K}}f\right)$, we use an argument analogous to that of Lemma 3. That is, we partition $k\mathbb{K}$ into $m-1$ disjoint Borel sets ${\mathrm{\Pi }}_m=\left\{{\mathbb{A}}_1,\dots ,{\mathbb{A}}_{m-1}\right\}$, and we approximate $g_k\star \left({\mathbf{1}}_{\mathbb{K}}f\right)$ by a $h_{m-1}^g\in {\mathcal{M}}_{m-1}^g$, where for each $i\in \left[m-1\right]$, $k_i=k$, $z_i\in {\mathbb{A}}_i$, and

$c_i=k^{-n}\int {\mathbf{1}}_{{\mathbb{A}}_i}f\left(\frac{z}{k}\right)\mathrm{d}\lambda \left(z\right).$

Define $k_m\in {\mathbb{R}}_{+}$, $z_m\in {\mathbb{R}}^n$, and $c_m=1-{\sum }_{i=1}^{m-1}{c}_i$, where

(20) $c_m=\int f\mathrm{d}\lambda -\int {\mathbf{1}}_{\mathbb{K}}f\mathrm{d}\lambda ={∥f-{\mathbf{1}}_{\mathbb{K}}f∥}_{{\mathcal{L}}_1}\le \frac{ϵ}{24}$(20)

by (16). Then, by a similar argument to Lemma 3, $c_i\ge 0$ for all $i\in \left[m\right]$ and ${\sum }_{i=1}^m{c}_i=1$. Thus, we may define an element $h_m^g\in {\mathcal{M}}_m^g$ via the parameters above.

For sufficiently large $k\ge k_2$, we use Lemma 3 to show that

${∥g_k\star \left({\mathbf{1}}_{\mathbb{K}}f\right)-h_{m-1}^g∥}_{{\mathcal{L}}_{\mathrm{\infty }}\left({\mathbb{K}}_k\right)}<\frac{ϵ}{8\left(\lambda \left(\mathbb{K}\right)+1\right)},$

which implies

${∥g_k\star \left({\mathbf{1}}_{\mathbb{K}}f\right)-h_{m-1}^g∥}_{{\mathcal{L}}_1\left({\mathbb{K}}_k\right)}<\int {\mathbf{1}}_{{\mathbb{K}}_k}\frac{ϵ}{8\left(\lambda \left(\mathbb{K}\right)+1\right)}\mathrm{d}\lambda$

(21) $<\frac{ϵ\lambda \left({\mathbb{K}}_k\right)}{8\left(\lambda \left(\mathbb{K}\right)+1\right)}<\frac{ϵ}{8},$(21)

and thus (19) is proved. Using (19), we write

${∥g_k\star \left({\mathbf{1}}_{\mathbb{K}}f\right)-h_m^g∥}_{{\mathcal{L}}_1}$

$={∥g_k\star \left({\mathbf{1}}_{\mathbb{K}}f\right)-h_{m-1}^g-c_m{k}_m^{n}g\left(k_{m}x-z_m\right)∥}_{{\mathcal{L}}_1}$

$\le {∥g_k\star \left({\mathbf{1}}_{\mathbb{K}}f\right)-h_{m-1}^g∥}_{{\mathcal{L}}_1\left({\mathbb{K}}_k\right)}$

$\text{+ left| {{g\_k}} right. star left({{{bf{1}}\_{mathbb K}f} right) - h\_{m - 1}^gleft| {\_{\_{{mathcal{L}\_1}left({mathbb K}\_k} right)}}} right.}$

$+{∥c_m{k}_m^{n}g\left(k_{m}x-z_m\right)∥}_{{\mathcal{L}}_1}$

$\le \frac{\text{isin}}{8}+c_m+∥{ }_{g_k}\star \left({\mathbf{1}}_{\mathbb{K}}f\right)∥{ }_{{\mathcal{L}}_1\left({\mathbb{K}}_k^c\right)}+∥h_{m-1}^g∥{ }_{{\mathcal{L}}_1\left({\mathbb{K}}_k^c\right)},$

where ${∥c_m{k}_m^{n}g\left(k_{m}x-z_m\right)∥}_{{\mathcal{L}}_1}\le c_m$ since $k_m^{n}g\left(k_{m}x-z_m\right)$ is a pdf. The aim is now to prove that

$\text{left| {{g\_k}} right. star left({{{bf{1}}\_mathbb{K}}f} right)left| {\_{{mathcal{L}\_1}left({{{mathbb K}\_k}} right)}} right. \&lt {epsilon over {24}},{rm{and}},h\_{m - 1}^gleft| {\_{{mathcal{L}\_1}left({{{mathbb K}\_k}} right)}} right. \&lt {epsilon over {24}}{rm{.}}}$

Using polar coordinates and (17), we have

$\int {\mathbf{1}}_{\left\{x:{∥x-y∥}_2>k^{-\gamma }\right\}}{g}_k\left(x-y\right)\mathrm{d}\lambda \left(x\right)$

$\le \int \frac{{\mathbf{1}}_{\left\{x:{∥x-y∥}_2>k^{-\gamma }\right\}}\beta k^{-\theta }}{{\left(k^{-1}+{∥x-y∥}_2\right)}^{n+\theta }}\mathrm{d}\lambda \left(x\right)$

$=\beta A_n{k}^{-\theta }\int \frac{{\mathbf{1}}_{\left(k^{-\gamma },\mathrm{\infty }\right)}{r}^{n-1}}{{\left(k^{-1}+r\right)}^{n+\theta }}\mathrm{d}\lambda \left(r\right)$

$\le \beta A_n{k}^{-\theta }\int {\mathbf{1}}_{\left(k^{-\gamma },\mathrm{\infty }\right)}{r}^{-\theta -1}\mathrm{d}\lambda \left(r\right)$

$=\beta A_n{k}^{\theta \left(\gamma -1\right)}/\theta ,$

where $A_n$ is the surface area of a unit sphere embedded in ${\mathbb{R}}^n$. We then have

$∥g_k\star \left({\mathbf{1}}_{\mathbb{K}}f\right)∥{ }_{{ }_{{\mathcal{L}}_1\left({\mathbb{K}}_k^c\right)}}$

$=\int \int {\mathbf{1}}_{\left\{y\in \mathbb{K}\right\}}{\mathbf{1}}_{\left\{x\in {\mathbb{K}}_k^c\right\}}f\left(y\right)g_k\left(x-y\right)\mathrm{d}\lambda \left(x\right)\mathrm{d}\lambda \left(y\right)$

$\le {∥{\mathbf{1}}_{\mathbb{K}}f∥}_{{\mathcal{L}}_{\mathrm{\infty }}}\int \int {\mathbf{1}}_{\left\{y\in \mathbb{K}\right\}}\frac{{\mathbf{1}}_{\left\{x:{∥x-y∥}_2>k^{-\gamma }\right\}}\beta k^{-\theta }}{{\left(k^{-1}+{∥x-y∥}_2\right)}^{n+\theta }}\mathrm{d}\lambda \left(x\right)\mathrm{d}\lambda \left(y\right)$

$\le {∥{\mathbf{1}}_{\mathbb{K}}f∥}_{{\mathcal{L}}_{\mathrm{\infty }}}\lambda \left(\mathbb{K}\right)\beta A_n{k}^{\theta \left(\gamma -1\right)}/\theta ,$

which implies that we can choose a $k_3\in \mathbb{N}$, such that for all $k\ge k_3$,

(22) $\text{left| {{g\_k}} right. star left({{{bf{1}}\_{mathbb{K}}f}} right)| {\_{\_{{mathcal{L}}\_1,left({{{mathbb K}\_k^c}} right), ,}lt,,{ epsilon over {24}}{rm{.}}}$(22)

Lastly, we write

$\text{left| {h\_{m - 1}^g} right.left| {\_{{mathcal{L}}\_1,left({mathbb K}\_k^{rm C}} right)}} right.}$

$=\int {\mathbf{1}}_{{\mathbb{K}}_{\mathrm{k}}^{\mathrm{C}}}\sum _{i=1}^{m-1}{c}_i{k}^{n}g\left(kx-z_i\right)\mathrm{d}\lambda$

$=\sum _{i=1}^{m-1}\int {\mathbf{1}}_{{\mathbb{K}}_{\mathrm{k}}^{\mathrm{C}}}\left[k^{-n}\int {\mathbf{1}}_{{\mathbb{A}}_{\mathrm{i}}}f\left(\frac{z}{k}\right)\mathrm{d}\lambda \left(z\right)\right]k^{n}g\left(kx-z_i\right)\mathrm{d}\lambda \left(x\right)$

$\le ∥{\mathbf{1}}_{\mathbb{K}}f∥{ }_{{\mathcal{L}}_{\mathrm{\infty }}}\sum _{i=1}^{m-1}{k}^{-n}\lambda \left({\mathbb{A}}_{\mathrm{i}}\right)\int {\mathbf{1}}_{{\mathbb{K}}_{\mathrm{k}}^{\mathrm{C}}}{g}_k\left(x-\frac{z_i}{k}\right)\mathrm{d}\lambda$

$\le {∥{\mathbf{1}}_{\mathbb{K}}f∥}_{{\mathcal{L}}_{\mathrm{\infty }}}\sum _{i=1}^{m-1}{k}^{-n}\lambda \left({\mathbb{A}}_i\right)\frac{\beta A_n{k}^{\theta \left(\gamma -1\right)}}{\theta }$

$\le {∥{\mathbf{1}}_{\mathbb{K}}f∥}_{{\mathcal{L}}_{\mathrm{\infty }}}\lambda \left(\mathbb{K}\right)\beta A_n{k}^{\theta \left(\gamma -1\right)}/\theta ,$

which implies that we can choose the same $k_3$ as above to obtain the bound

(23) $\text{| {h\_{m - 1}^g} | {\_{\_{{mathcal{L}}\_1,left({{{mathbb K}\_k^c}} right)} lt, { {epsilon} over {24}}{rm{,}}}$(23)

for any $k\ge k_3$.

Thus, we obtain the bound ${∥{\mathbf{1}}_{\mathbb{K}}f-h_m^g∥}_{{\mathcal{L}}_1}<ϵ/2$, for all $k\ge max\left\{k_1,k_2,k_3\right\}$, by combining (18), (19), (20), (21), (22), and (23), via the triangle inequality. The result is proved by combing the bound above, with (16), for an appropriately large $r\left(ϵ\right)\in \mathbb{N}$.

3. Comments and discussion

3.1. Relationship to Theorem 1

In the proof of Theorem 1, the famous Hilbert space approximation result of Jones (1992) and Barron (1993) was used to bound the ${\mathcal{L}}_2$ norm between any approximand $f\in {\mathcal{L}}_2$ and a convex combination of bounded functions in ${\mathcal{L}}_2$. This approximation theorem is exactly the $p=2$ case of the more general theorem of Donahue et al. (1997), as presented in Lemma 11. Thus, one can view Theorem 5(c) as the $p\in \left(1,\mathrm{\infty }\right)$ generalization of Theorem 1.

3.2. The class $\mathcal{W}$ is a proper subset of the class ${\mathcal{C}}_0$

Here, we comment on the nature of class $\mathcal{W}$, which was investigated by Bacharoglou, (2010) and Nestoridis et al. (2011). We recall that Bacharoglou (2010) conjectured that Theorem 4 generalizes from $g=\varphi$ to $g\in \mathcal{V}$. In Theorem 5(a)–(e), we assume that $g\in {\mathcal{C}}_0$. We can demonstrate that $g\in {\mathcal{C}}_0$ is a strictly weaker condition than $g\in \mathcal{V}$ or $g\in \mathcal{W}$.

For example, consider the function in $g:\mathbb{R}\to \mathbb{R}$ such that $g\left(x\right)=0$ if $x<0$ and

$g\left(x\right)=\sum _{i=1}^{\mathrm{\infty }}\frac{2^{2i}}{i}\left[{\left(x-i+1\right)}^{2i}{\mathbf{1}}_{\left\{i-1\le x<i-1/2\right\}}+{\left(x-i\right)}^{2i}{\mathbf{1}}_{\left\{i-1/2\le x<i\right\}}\right]\mathrm{i}\mathrm{f}x\ge 0,$

and note that

$\int {\mathbf{1}}_{\left(-1/2,1/2\right)}\frac{{\left(2x\right)}^{2i}}{i}\mathrm{d}\lambda =\frac{1}{2i^2+i}<\frac{1}{i^2}.$

Since ${\sum }_{i=1}^{\mathrm{\infty }}\left(1/i^2\right)={\pi }^2/6$, $g\in {\mathcal{L}}_1$. Furthermore, $g$ is continuous since all stationary points of $g$ are continuous. In $\mathbb{R}$, $g\in {\mathcal{C}}_0$ if

$\underset{x\to \pm \mathrm{\infty }}{lim}g\left(x\right)=0.$

For $x\le 0$, we observe that $g=0$ and thus the left limit is satisfied. On the right, for any $1/ϵ>0$, we have $x\left(ϵ\right)\ge ⌈ϵ⌉-1/2$, so that $g\left(x\right)<1/ϵ$, for all $x>x\left(ϵ\right)$, where $⌈\cdot ⌉$ is the ceiling operator. Therefore, $g\in {\mathcal{C}}_0$.

Within each interval $i-1\le x<i$, we observe that $g$ is locally maximized at $x=i-1/2$. The local maximum corresponding to each of these points is $1/i$. Thus $g\notin \mathcal{W}$, since

$\sum _{i=1}^{\mathrm{\infty }}\frac{1}{i}<\sum _{y\in \mathbb{Z}}\underset{x\in \left[0,1\right]}{sup}\left|g\left(x+y\right)\right|,$

where ${\sum }_{i=1}^{\mathrm{\infty }}\left(1/i\right)=\mathrm{\infty }$. Furthermore, $g\notin \mathcal{V}$ since $\mathcal{V}\subset \mathcal{W}$.

3.3. Convergence in measure

Along with the conclusions of Theorem 5(d) and (e), Lemma 12 also implies convergence in measure. That is, if $\nu$ is a $\sigma -\mathrm{f}\mathrm{i}\mathrm{n}\mathrm{i}\mathrm{t}\mathrm{e}$ Borel measure on ${\mathbb{R}}^n$, then for any $\nu -\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{b}\mathrm{l}\mathrm{e}$ $f$, there exists a sequence $\left\{h_m^g\right\}$, such that for any $ϵ>0$,

$\underset{m\to \mathrm{\infty }}{lim}\upsilon \left(\left\{x\in {\mathbb{R}}^n:\left|f\left(x\right)-h_m^g\left(x\right)\right|\ge ϵ\right\}\right)=0.$

A Technical results

Throughout the main text, we utilize a number of established technical results. For the convenience of the reader, we append these results within this Appendix. Sources from which we draw the unproved results are provided at the end of the section.

Lemma 5. Let $\left\{g_k\right\}$ be a sequence of pdfs in ${\mathcal{L}}_1$ and for every $\delta >0$

$\underset{k\to \mathrm{\infty }}{lim}\int {\mathbf{1}}_{\left\{x:{∥x∥}_2>\delta \right\}}{g}_k\mathrm{d}\lambda =0.$

Then, for all $f\in {\mathcal{L}}_p$ and $1\le p<\mathrm{\infty }$,

$\underset{k\to \mathrm{\infty }}{lim}{∥g_k\star f-f∥}_{{\mathcal{L}}_p}=0.$

Furthermore, for all $f\in {\mathcal{C}}_b$ and any compact $\mathbb{K}\subset {\mathbb{R}}^n$,

$\underset{k\to \mathrm{\infty }}{lim}{∥g_k\star f-f∥}_{{\mathcal{L}}_{\mathrm{\infty }}\left(\mathbb{K}\right)}=0.$

The sequences $\left\{g_k\right\}$ from Lemma 5 are often called approximate identities or approximations of the identity. A simple construction of approximate identities is by taking dilations $g_k\left(x\right)=k^{n}g\left(kx\right)$, which yields the following corollary.

Corollary 1. Let $g$ be a pdf. Then the sequence of dilations $\left\{g_k:g_k\left(x\right)=k^{n}g\left(kx\right)\right\}$, satisfies the hypothesis of Lemma 5 and hence permits its conclusion.

Lemma 6. The class ${\mathcal{C}}_0$ is a subset of ${\mathcal{C}}_b$. Furthermore, if $f\in {\mathcal{C}}_0$, then $f$ is uniformly continuous.

Lemma 7 (Urysohn’s Lemma). If $\mathbb{K}\subset {\mathbb{R}}^n$ is compact, then there exists some $g\in {\mathcal{C}}_c$, such that $0\le g\le 1$ and ${\mathbf{1}}_{\mathbb{K}}g=1$.

Lemma 8. If $\mathbb{X}\subset {\mathbb{R}}^n$ is bounded, then for any $r>0$, $\mathbb{X}$ can be covered by ${\bigcup }_{i=1}^m\mathbb{B}\left(x_i,r\right)$ for some finite $m\in \mathbb{N}$, where $x_i\in {\mathbb{R}}^n$ and $i\in \left[m\right]$.

Lemma 9. If $0<p<q<r\le \mathrm{\infty }$, then ${\mathcal{L}}_p\cap {\mathcal{L}}_r\subset {\mathcal{L}}_q$.

Let $\mathrm{\Gamma }:\mathbb{R}\to \mathbb{R}$ be the usual gamma function, defined as $\mathrm{\Gamma }\left(z\right)=\int {\mathbf{1}}_{\left(0,\mathrm{\infty }\right)}{x}^{z-1}exp\left(-x\right)\mathrm{d}\lambda$.

Lemma 10. If $f\in {\mathcal{L}}_p$ and $g\in {\mathcal{L}}_1$, for $1\le p\le \mathrm{\infty }$, then $f\star g$ exists and we have ${∥f\star g∥}_{{\mathcal{L}}_p}\le {∥g∥}_{{\mathcal{L}}_1}{∥f∥}_{{\mathcal{L}}_p}$.

Lemma 11. Let $\mathcal{G}\subset {\mathcal{L}}_p$, for some $1\le p<\mathrm{\infty }$, and let $f\in \overline{Conv}\left(\mathcal{G}\right)$. For any $K>0$, such that ${∥f-\alpha ∥}_{{\mathcal{L}}_p}<K$, for all $\alpha \in \mathcal{G}$, there exists a $h_m\in Con{v}_m\left(\mathcal{G}\right)$, such that

${∥f-h_m∥}_{{\mathcal{L}}_p}\le \frac{C_{p}K}{m^{1-1/\alpha }},$

where $\alpha =min\left\{p,2\right\}$, and

$C_p=\left(\begin{array}{cc}1& if1\le p\le 2,\\ \sqrt{2}{\left[\sqrt{\pi }\mathrm{\Gamma }\left(\frac{p+1}{2}\right)\right]}^{1/p}& ifp>2.\end{array}\right.$

Lemma 12. In any measure $\nu$, uniform convergence implies almost uniform convergence, and almost uniform convergence implies almost everywhere convergence and convergence in measure, with respect to $\nu$.

B Sources of results

Lemma 5 is reported as Theorem 9.3.3 in Makarov & Podkorytov (2013) (see also Theorem 2 of Rudin, 1976, Ch. 20). The proof of Corollary 1 can be taken from that of Theorem 4 of Cheney & Light, 2000, Ch. 20. Lemma 6 appears in Conway, 2012), as Proposition 1.4.5. Lemma 7 is taken from Corollary 1.2.9 of Conway (2012). Lemma 8 appears as Theorem 1.2.2 in Conway (2012). Lemma 9 can be found in Folland (1999), Prop. 6.10. Lemma 10 can be found in [21, Thm. 9.3.1]. Lemma 11 appears as Corollary 2.6 in Donahue et al. (1997). Lemma 12 can be obtained from the definition of almost uniform convergence, Lemma 7.10, and Theorem 7.11 of Bartle (1995).

Acknowledgements

HDN is personally funded by Australian Research Council (ARC) grant DE170101134. HDN and GJM are supported by ARC grant DP180101192. FC is supported by Agence Nationale de la Recherche (ANR) grant SMILES ANR-18-CE40-0014 and by Région Normandie grant RIN.

Additional information

Notes on contributors

Geoffrey J. McLachlan

Mr. Nguyen, Dr. Nguyen, and Profs. Chamroukhi and McLachlan are each keenly interested in the study of finite mixture models and their modifications and extensions for various data analytic and machine learning applications. Their research spans the fields of computational statistics, mathematical statistics, machine learning and AI. Common threads of their research consist of the derivation of algorithms for the efficient estimation of mixture models and extensions, such as mixtures of experts, and mixtures of factor analysers; the derivation of theoretical results regarding the statistical and mathematical properties of such constructions and their estimators; and the application of mixture models to data analytic problems spanning the fields of biology, medical science, engineering, signal processing, among many others.