Robust equivariant non parametric regression estimators for functional ergodic data

Abstract

This article deals with the equivariant non parametric robust regression estimation for stationary ergodic processes valued in $\mathcal{F}\times \mathbb{R},$ where $\mathcal{F}$ is a semi-metric space. We consider a new robust regression estimator when the scale parameter is unknown. The principal aim is to prove the almost complete convergence (with rate) for the proposed estimator. Unlike in standard multivariate cases, the gap between pointwise and uniform results is not immediate. So, suitable topological considerations are needed, implying changes in the rates of convergence which are quantified by entropy considerations.

Keywords:

• Functional data
• ergodic data
• scale parameter
• kernel estimate
• almost complete convergence
• entropy

Acknowledgments

The authors would like to thank the anonymous reviewers for their valuable comments and suggestions which improved substantially the quality of this article. The authors extend their appreciation to the Deanship of Scientific Research at King Khalid University for funding this work through General Research Project under grant number (G.R.P-125-40).

Appendix

The proof of Lemma $4.1$ is very close to that of Lemma $\mathbf{A}.4.$ in Boente and Vahnovan (2015).

According to $(\mathbf{H}1)$ and $(\mathbf{H}4),$ it is clear that if $K(1)>C>0,$ (A1) $$\forall x\in S_{\mathcal{F}},\hspace{1em}\exists 0<C<C^{\prime }<\infty ,\hspace{1em}C\varphi \left(h\right)<{\mathbb{E}}^{{\mathcal{F}}_{i-1}}\left[K_1\left(x\right)\right]<C^{\prime }\varphi \left(h\right)$$(A1)

In the situation when $K(1)=0,$ the combination of $(\mathbf{H}1)$ and $(\mathbf{H}5)(\mathbf{i})$ allows to get the same result. From now on, in order to simplify the notation, we set $ϵ=\frac{\hspace{0.17em}\text{log}\hspace{0.17em}n}{n}.$

Proof of Theorem 4.2.

We consider the decomposition: (A2) $$\begin{array}{c}\widehat{\lambda }\left(x,a,\widehat{s}\left(x\right)\right)-\lambda \left(x,a,\widehat{s}\left(x\right)\right)=\frac{1}{{\widehat{\lambda }}_D\left(x\right)}\left[{\widehat{\lambda }}_N\left(x,a,\widehat{s}\left(x\right)\right)-{\overline{\lambda }}_N\left(x,a,\widehat{s}\left(x\right)\right)\right]\\ +\frac{1}{{\widehat{\lambda }}_D\left(x\right)}\left[{\overline{\lambda }}_N\left(x,a,\widehat{s}\left(x\right)\right)-\lambda \left(x,a,\widehat{s}\left(x\right)\right)\right]+\left[1-{\widehat{\lambda }}_D\left(x\right)\right]\frac{\lambda \left(x,a,\widehat{s}\left(x\right)\right)}{{\widehat{\lambda }}_D\left(x\right)}\end{array}$$(A2) where $$\begin{array}{c}{\widehat{\lambda }}_D\left(x\right):=\frac{1}{n\mathbb{E}\left[K(h^{-1}d\left(x,X_1\right))\right]}\sum _{i=1}^{n}K\left(h^{-1}d\left(x,X_i\right)\right),\\ {\overline{\lambda }}_D\left(x\right):=\frac{1}{n\mathbb{E}\left[K(h^{-1}d\left(x,X_1\right))\right]}\sum _{i=1}^n{\mathbb{E}}^{{\mathcal{F}}_{i-1}}\left[K\left(h^{-1}d\left(x,X_i\right)\right)\right],\\ {\widehat{\lambda }}_N\left(x,a,\widehat{s}\left(x\right)\right):=\frac{1}{n\mathbb{E}\left[K(h^{-1}d\left(x,X_1\right))\right]}\sum _{i=1}^{n}K\left(h^{-1}d\left(x,X_i\right)\right){\psi }_x\left(\frac{Y_i-a}{\widehat{s}\left(x\right)}\right),\\ {\overline{\lambda }}_N\left(x,a,\widehat{s}\left(x\right)\right):=\frac{1}{n\mathbb{E}\left[K(h^{-1}d\left(x,X_1\right))\right]}\sum _{i=1}^n{\mathbb{E}}^{{\mathcal{F}}_{i-1}}\left[K\left(h^{-1}d\left(x,X_i\right)\right){\psi }_x\left(\frac{Y_i-a}{\widehat{s}\left(x\right)}\right)\right]\end{array}$$

Therefore, Theorem 4.2 is a consequence of the following intermediate results.

Lemma A.1.

Under hypotheses $(\mathbf{H}1)$ and $(\mathbf{H}4)$-$(\mathbf{H}6)$, we have $$\underset{x\in S_{\mathcal{F}}}{\sup }\left|{\widehat{\lambda }}_D\left(x\right)-1\right|=O_{a.co.}\left(\sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(\hspace{0.17em}\text{log}\hspace{0.17em}\left(n\right)/n\right)}{n\varphi \left(h\right)}}\right)$$

Corollary A.2.

Under the hypotheses of Lemma A.1, we have $$\sum _{n=1}^{\infty }\mathbb{P}\left(\underset{x\in S_{\mathcal{F}}}{\inf }{\widehat{\lambda }}_D(x)<\frac{1}{2}\right)<\infty$$

Lemma A.3.

Under the hypotheses $(\mathbf{H}1),(\mathbf{H}2)$ and $(\mathbf{H}4)-(\mathbf{H}6)$, we have $$\underset{x\in S_{\mathcal{F}}}{\sup }\left|{\overline{\lambda }}_N\left(x,a,\widehat{s}\left(x\right)\right)-\lambda \left(x,a,\widehat{s}\left(x\right)\right)\right|=O\left(h^{{\eta }_1}\right)$$

Lemma A.4.

Under the assumptions of Theorem 4.2, we have $$\underset{x\in S_{\mathcal{F}}}{\sup }\left|{\widehat{\lambda }}_N\left(x,a,\widehat{s}\left(x\right)\right)-{\overline{\lambda }}_N\left(x,a,\widehat{s}\left(x\right)\right)\right|=O\left(\sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(\hspace{0.17em}\text{log}\hspace{0.17em}\left(n\right)/n\right)}{n\varphi \left(h\right)}}\right)\hspace{1em}a.co$$

Proof of Lemma A.1.

Let $x_1,\dots ,x_{N_{ϵ}(S_{\mathcal{F}})}$ be an $ϵ-$ net for $S_{\mathcal{F}}$ and for all $x\in S_{\mathcal{F}},$ one sets $k(x)=\mathit{arg}{\min }_{k\in \left\{1,\dots ,N_{ϵ}(S_{\mathcal{F}})\right\}}d(x,x_k).$ One considers the following decomposition: $$\begin{array}{c}\underset{x\in S_{\mathcal{F}}}{\sup }\left|{\widehat{\lambda }}_D(x)-{\overline{\lambda }}_D(x)\right|\le \underset{F_1}{\underbrace{\underset{x\in S_{\mathcal{F}}}{\sup }\left|{\widehat{\lambda }}_D(x)-{\widehat{\lambda }}_D(x_{k(x)})\right|}}+\underset{F_2}{\underbrace{\underset{x\in S_{\mathcal{F}}}{\sup }\left|{\widehat{\lambda }}_D(x_{k(x)})-{\overline{\lambda }}_D(x_{k(x)})\right|}}\\ +\underset{F_3}{\underbrace{\underset{x\in S_{\mathcal{F}}}{\sup }\left|{\overline{\lambda }}_D(x_{k(x)})-{\overline{\lambda }}_D(x)\right|}}\end{array}$$

• Let us study F₁. By using (A1) and the boundness of K, one can write $$\begin{array}{c}{F}_1\le \underset{x\in S_{\mathcal{F}}}{\sup }\frac{1}{n}\sum _{i=1}^n\left|\frac{1}{\mathbb{E}\left[K_1(x)\right]}{K}_i(x)-\frac{1}{\mathbb{E}\left[K_1(x_{k(x)})\right]}{K}_i(x_{k(x)})\right|\\ \le \frac{C}{\varphi (h)}\underset{x\in S_{\mathcal{F}}}{\sup }\frac{1}{n}\sum _{i=1}^n\left|K_i(x)-K_i(x_{k(x)})\right|{\mathbb{I}}_{B(x,h)\cup B(x_{k(x)},h)}(X_i)\end{array}$$

Let us first consider the case $K(1)=0.$ Because K is Lipschitz on $\left[0,1\right]$ in this case, it comes $$F_1\le \underset{x\in S_{\mathcal{F}}}{\sup }\frac{C}{n}\sum _{i=1}^n{Z}_i\text{with}{Z}_i=\frac{ϵ}{h\varphi \left(h\right)}{\mathbb{I}}_{B\left(x,h\right)\cup B(x_{k\left(x\right)},h)}\left(X_i\right)$$ with, uniformly on x, $$Z_1=O\left(\frac{ϵ}{h\varphi \left(h\right)}\right),\mathbb{E}\left[Z_1\right]=O\left(\frac{ϵ}{h}\right)\text{and}\mathit{Var}\left(Z_1\right)=O\left(\frac{{ϵ}^2}{h^2\varphi \left(h\right)}\right)$$

A standard inequality for sums of bounded random variables with $(\mathbf{H}5)(\mathbf{ii})$ allows to get $$F_1=O\left(\frac{ϵ}{h}\right)+O_{a.co}\left(\frac{ϵ}{h}\sqrt{\frac{\hspace{0.17em}\text{log}\hspace{0.17em}n}{n\varphi \left(h\right)}}\right)$$ and it suffices to combine $(\mathbf{H}5)(\mathbf{i})$ and $(\mathbf{H}5)(\mathbf{ii})$ to get $$F_1=O_{a.co}\left(\sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)$$

Now, let $K(1)>C>0.$ In this situation K is Lipschitz on $[0,1).$ One has to decompose F₁ into three terms as follows: $$F_1\le C\underset{x\in S_{\mathcal{F}}}{\sup }\left(F_{11}+F_{12}+F_{13}\right)$$ with $$\begin{array}{c}{F}_{11}=\frac{1}{n\varphi (h)}\sum _{i=1}^n\left|K_i(x)-K_i(x_{k(x)})\right|{\mathbb{I}}_{B(x,h)\cap B(x_{k(x)},h)}(X_i),\\ F_{12}=\frac{1}{n\varphi (h)}\sum _{i=1}^n{K}_i(x){\mathbb{I}}_{B(x,h)\cap \overline{B}(x_{k(x)},h)}(X_i),\\ F_{13}=\frac{1}{n\varphi (h)}\sum _{i=1}^n{K}_i(x_{k(x)}){\mathbb{I}}_{\overline{B}(x,h)\cap B(x_{k(x)},h)}(X_i)\end{array}$$

One can follow the same steps (i.e., case $K(1)=0$) for studying F₁₁ and one gets the same result: $$F_{11}=O_{a.co.}\left(\sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)$$

Following same ideas for studying $F_{12},$ one can write $F_{12}\le \frac{C}{n}{\sum }_{i=1}^n{W}_i$ with $W_i=\frac{1}{\varphi (h)}{\mathbb{I}}_{B(x,h)\cap \overline{B}(x_{k(x)}h)}(X_i),$ and by using again $(\mathbf{H}5)(\mathbf{i})$ and the same inequality for sums of bounded random variables, we get $$F_{12}=O\left(\frac{ϵ}{\varphi \left(h\right)}\right)+O_{a.co}\left(\sqrt{\frac{ϵ\hspace{0.17em}\text{log}\hspace{0.17em}n}{n\varphi {\left(h\right)}^2}}\right)$$

Similarly, one can state the same rate of convergence for $F_{13}.$ To end the study of $F_1,$ it suffices to put together all the intermediate results and to use $(\mathbf{H}5)(\mathbf{ii})$ for getting $$F_1=O_{a.co}\left(\sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)$$

• Now, concerning $F_2,$ we have, for all $\eta >0,$ $$\begin{array}{l}\mathbb{P}\left(F_2>\eta \sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)\\ =\mathbb{P}\left(\underset{{}_{k\in \left\{1,\dots ,N_{ϵ}\left(S_{\mathcal{F}}\right)\right\}}}{\max }\left|{\widehat{\lambda }}_D\left(x_k\right)-{\overline{\lambda }}_D\left(x_k\right)\right|>\eta \sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)\\ \le N_{ϵ}\left(S_{\mathcal{F}}\right)\underset{{}_{k\in \left\{1,\dots ,N_{ϵ}\left(S_{\mathcal{F}}\right)\right\}}}{\max }\mathbb{P}\left(\left|{\widehat{\lambda }}_D\left(x_k\right)-{\overline{\lambda }}_D\left(x_k\right)\right|>\eta \sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right).\\ {\widehat{\lambda }}_D\left(x_k\right)-{\overline{\lambda }}_D\left(x_k\right)=\frac{1}{n\mathbb{E}\left[K\left(\frac{d\left(x,X_1\right)}{h}\right)\right]}\sum _{i=1}^n{\Lambda }_{ki}\left(x_k\right)\end{array}$$

with $${\Lambda }_{ki}\left(x_k\right)=K_i\left(x_k\right)-{\mathbb{E}}^{{\mathcal{F}}_{i-1}}\left[K_i\left(x_k\right)\right]$$

Using the fact that ${\Lambda }_{ki}(x_k)$ is a triangular array of martingale differences according to the σ-fields ${({\mathcal{F}}_{i-1})}_i$ to apply the inequality of Lemma 1 in Laïb and Louani (2011). To do that, we must evaluate the quantity ${\mathbb{E}}^{{\mathcal{F}}_{i-1}}\left[{\Lambda }_{ki}^p(x)\right].$

Indeed, under $(\mathbf{H}1)$ and $(\mathbf{H}4)$ we have, for $p\ge 2$ $$\begin{array}{c}{\mathbb{E}}^{{\mathcal{F}}_{i-1}}\left[{\Lambda }_{ki}^p\left(x\right)\right]<C{\mathbb{E}}^{{\mathcal{F}}_{i-1}}\left[{\Lambda }_{ki}^2\left(x\right)\right]\\ \le C{\mathbb{E}}^{{\mathcal{F}}_{i-1}}\left[K_i^2\left(x\right)\right]\\ <C\mathbb{P}\left(X_i\in B\left(x,h\right)|{\mathcal{F}}_{i-1}\right)\\ \le C{\varphi }_i\left(h\right)\end{array}$$

Hence, $$\begin{array}{c}\mathbb{P}\left(\left|{\widehat{\lambda }}_D\left(x_k\right)-{\overline{\lambda }}_D\left(x_k\right)\right|>\eta \sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)=\mathbb{P}\left(\frac{1}{n}\left|\sum _{i=1}^n{\Lambda }_{ki}\right|>\eta \sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)\\ \le 2\hspace{0.17em}\text{exp}\hspace{0.17em}\left\{-C{\eta }^2{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)\right\}\end{array}$$

Thus, by using the fact that ${\Gamma }_{S_{\mathcal{F}}}(ϵ)=\hspace{0.17em}\text{log}\hspace{0.17em}{N}_{ϵ}(S_{\mathcal{F}})$ and by choosing η such that $C{\eta }^2=\beta ,$ we have (A3) $$N_{ϵ}\left(S_{\mathcal{F}}\right)\underset{{}_{k\in \left\{1,\dots ,N_{ϵ}\left(S_{\mathcal{F}}\right)\right\}}}{\max }\mathbb{P}\left(\left|{\widehat{\lambda }}_D\left(x_k\right)-{\overline{\lambda }}_D\left(x_k\right)\right|>\eta \sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)\le C^{\prime }{N}_{ϵ}{\left(S_{\mathcal{F}}\right)}^{1-\beta }$$(A3)

Because ${\sum }_{n=1}^{\infty }{N}_{ϵ}{(S_{\mathcal{F}})}^{1-\beta }<\infty ,$ we obtain that $F_2=O_{a.co}(\sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}(ϵ)}{n\varphi (h)}}).$

• For $F_3,$ it is clear that $F_3\le {\mathbb{E}}^{{\mathcal{F}}_{i-1}}(\underset{x\in S_{\mathcal{F}}}{\sup }\left|{\widehat{\lambda }}_D(x)-{\widehat{\lambda }}_D(x_k)\right|)$ and by following a similar proof to the one used for studying $F_1,$ it comes $$F_3=O_{a.co}\left(\sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)$$

Proof of Corollary A.2.

It is easy to see that

$\underset{x\in S_{\mathcal{F}}}{\inf }\left|{\widehat{\lambda }}_D(x)\right|\le \frac{1}{2}\Rightarrow \exists x\in S_{\mathcal{F}}$ such that $1-{\widehat{\lambda }}_D(x)\ge \frac{1}{2}\Rightarrow \underset{x\in S_{\mathcal{F}}}{\sup }\left|1-{\widehat{\lambda }}_D(x)\right|\ge \frac{1}{2}.$

We deduce from Lemma A.1 that $$\mathbb{P}\left(\underset{x\in S_{\mathcal{F}}}{\inf }\left|{\widehat{\lambda }}_D\left(x\right)\right|\le \frac{1}{2}\right)\le \mathbb{P}\left(\underset{x\in S_{\mathcal{F}}}{\sup }\left|1-{\widehat{\lambda }}_D\left(x\right)\right|\ge \frac{1}{2}\right)$$

Consequently, $$\sum _{n=1}^{\infty }\mathbb{P}\left(\underset{x\in S_{\mathcal{F}}}{\inf }\left|{\widehat{\lambda }}_D\left(x\right)\right|\le \frac{1}{2}\right)<\infty$$

Proof of Lemma A.3.

One has $$\begin{array}{l}\left|{\overline{\lambda }}_N\left(x,a,\widehat{s}\left(x\right)\right)-\lambda \left(x,a,\widehat{s}\left(x\right)\right)\right|\\ =\left|\frac{1}{n\mathbb{E}\left[K_1\left(x\right)\right]}{\mathbb{E}}^{{\mathcal{F}}_{i-1}}\left[\sum _{i=1}^n{K}_i\left(x\right){\psi }_x\left(\frac{Y_i-a}{\widehat{s}\left(x\right)}\right)\right]-\lambda \left(x,a,\widehat{s}\left(x\right)\right)\right|\\ \le \left|\frac{1}{\mathbb{E}\left[K_1\left(x\right)\right]}{\mathbb{E}}^{{\mathcal{F}}_{i-1}}\left[K_1\left(x\right){\psi }_x\left(\frac{Y_1-a}{\widehat{s}\left(x\right)}\right)\right]-\lambda \left(x,a,\widehat{s}\left(x\right)\right)\right|\\ \le \frac{1}{\mathbb{E}\left[K_1\left(x\right)\right]}\left[{\mathbb{E}}^{{\mathcal{F}}_{i-1}}\left[K_1\left(x\right)\left|\lambda \left(X_1,a,\widehat{s}\left(x\right)\right)-\lambda \left(x,a,\widehat{s}\left(x\right)\right)\right|\right]\right]\end{array}$$

Hence, we get $$\forall x\in S_{\mathcal{F}},\left|{\overline{\lambda }}_N(x,a,\widehat{s}(x))-\lambda (x,a,\widehat{s}(x))\right|\le \frac{1}{\mathbb{E}\left[K_1(x)\right]}\left[{\mathbb{E}}^{{\mathcal{F}}_{i-1}}\left[K_1(x)\left|\lambda (X_1,a,\widehat{s}(x))-\lambda (x,a,\widehat{s}(x))\right|\right]\right]$$ Thus, with hypotheses $(\mathbf{H}1),(\mathbf{H}2)$ and (A1) we have $$\begin{array}{l}\forall x\in S_{\mathcal{F}},\left|{\overline{\lambda }}_N\left(x,a,\widehat{s}\left(x\right)\right)-\lambda \left(x,a,\widehat{s}\left(x\right)\right)\right|\\ \le C\frac{1}{\mathbb{E}\left[K_1\left(x\right)\right]}\left[{\mathbb{E}}^{{\mathcal{F}}_{i-1}}{K}_1\left(x\right){\mathbb{I}}_{B\left(x,h\right)}\left(X_1\right)d^{{\eta }_1}\left(X_1,x\right)\right]\le C{h}^{{\eta }_1}\end{array}$$ the proof of this lemma is then completed.

Proof of Lemma A.4.

This proof follows the same steps as the proof of Lemma A.1. For this, we keep these notations and we use the following decomposition: $$\begin{array}{c}\underset{x\in S_{\mathcal{F}}}{\sup }\left|{\widehat{\lambda }}_N\left(x,a,\widehat{s}\left(x\right)\right)-{\overline{\lambda }}_N\left(x,a,\widehat{s}\left(x\right)\right)\right|\le \underset{G_1}{\underbrace{\underset{x\in S_{\mathcal{F}}}{\sup }\left|{\widehat{\lambda }}_N\left(x,a,\widehat{s}\left(x\right)\right)-{\widehat{\lambda }}_N\left(x_{k\left(x\right)},a,\widehat{s}\left(x\right)\right)\right|}}\\ +\underset{G_2}{\underbrace{\underset{x\in S_{\mathcal{F}}}{\sup }\left|{\widehat{\lambda }}_N\left(x_{k\left(x\right)},a,\widehat{s}\left(x\right)\right)-{\overline{\lambda }}_N\left(x_{k\left(x\right)},a,\widehat{s}\left(x\right)\right)\right|}}\\ +\underset{G_3}{\underbrace{\underset{x\in S_{\mathcal{F}}}{\sup }\left|{\overline{\lambda }}_N\left(x_{k\left(x\right)},a,\widehat{s}\left(x\right)\right)-{\overline{\lambda }}_N\left(x,a,\widehat{s}\left(x\right)\right)\right|}}\end{array}$$

Condition $(H1)$ and result (A1) allow to write directly, for G₁ and $G_3:$ $$\begin{array}{c}{G}_1=\underset{x\in S_{\mathcal{F}}}{\sup }\frac{1}{n}\sum _{i=1}^n\left|\frac{1}{\mathbb{E}\left[K_1(x)\right]}{K}_i(x){\psi }_x(\frac{Y_i-a}{\widehat{s}(x)})-\frac{1}{\mathbb{E}\left[K_1(x_{k(x)})\right]}{K}_i(x_{k(x)}){\psi }_x(\frac{Y_i-a}{\widehat{s}(x)})\right|\\ \le \underset{x\in S_{\mathcal{F}}}{\sup }\frac{1}{n\varphi (h)}\sum _{i=1}^n\left|{\psi }_x(\frac{Y_i-a}{\widehat{s}(x)})\right|\left|K_i(x)-K_i(x_{k(x)})\right|{\mathbb{I}}_{B(x,h)\cup B(x_{k(x)},h)}(X_i)\end{array}$$

Now, as for $F_1,$ one considers $K(1)=0$ (i.e., K Lipschitz on $\left[0,1\right]$) and one gets $G_1\le \frac{C}{n}{\sum }_{i=1}^n{Z}_i$ with $Z_i=\frac{ϵ{\psi }_x(\frac{Y_i-a}{\widehat{s}(x)})}{h\varphi (h)}\underset{x\in S_{\mathcal{F}}}{\sup }{\mathbb{I}}_{B(x,h)\cup B(x_{k(x)},h)}.$

The main difference with the study of F₁ is that one uses here an inequality for unbounded variables. Note that one has $$\mathbb{E}\left[{\psi }_x^j\left(\frac{Y-a}{\widehat{s}\left(x\right)}\right)\right]=\mathbb{E}\left[\mathbb{E}\left[{\psi }_x^j\left(\frac{Y-a}{\widehat{s}\left(x\right)}\right)|X\right]\right]=\int \delta \left(x\right)d{P}_X<C<\infty$$ which implies that $$\mathbb{E}\left({\left|Z_1\right|}^j\right)\le \frac{C{ϵ}^j}{h^j\varphi {\left(h\right)}^{j-1}}$$

Now, $(\mathbf{H}5)(\mathbf{ii})$ allows to get (A4) $$G_1=O_{a.co}\left(\sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)$$(A4)

If one considers the case $K(1)>C>0$ (i.e., K Lipschitz on $[0,1)$), one has to split G₁ into three terms as for F₁ and by using similar arguments, one can state the same rate of almost complete convergence. Similar steps allow to get (A5) $$G_3=O\left(\sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)$$(A5)

For $G_2,$ similarly to the proof of Lemma A.1, we have, $\forall \eta >0,$ $$\begin{array}{l}\mathbb{P}\left(G_2>\eta \sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)\\ =\mathbb{P}\left(\underset{{}_{k\in \left\{1,\dots ,N_{ϵ}\left(S_{\mathcal{F}}\right)\right\}}}{\max }\left|{\widehat{\lambda }}_N\left(x_k,a,\widehat{s}\left(x\right)\right)-{\overline{\lambda }}_N\left(x_k,a,\widehat{s}\left(x\right)\right)\right|>\eta \sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)\\ \le N_{ϵ}\left(S_{\mathcal{F}}\right)\underset{{}_{k\in \left\{1,\dots ,N_{ϵ}\left(S_{\mathcal{F}}\right)\right\}}}{\max }\mathbb{P}\left(\left|{\widehat{\lambda }}_N\left(x_k,a,\widehat{s}\left(x\right)\right)-{\overline{\lambda }}_N\left(x_k,a,\widehat{s}\left(x\right)\right)\right|>\eta \sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)\end{array}$$

The rest of the proof is based on the application of the exponential inequality of Lemma 1 in Laïb and Louani (2011) on the triangular array of martingale differences.

Indeed, let $${\Lambda }_{ki}=\frac{1}{\mathbb{E}\left[K_1\left(x_k\right)\right]}\left(K_i\left(x_k\right){\psi }_x\left(\frac{Y_i-a}{\widehat{s}\left(x\right)}\right)-{\mathbb{E}}^{{\mathcal{F}}_{i-1}}\left[K_i\left(x_k\right){\psi }_x\left(\frac{Y_i-a}{\widehat{s}\left(x\right)}\right)\right]\right)$$

For this, we must evaluate the quantity ${\mathbb{E}}^{{\mathcal{F}}_{i-1}}\left[{\Lambda }_{ki}^p\right].$ The latter can be evaluated by using the same arguments as were invoked for proving Lemma 5 in Laïb and Louani (2011), allowing us to write, under $(\mathbf{H}3),$ $${\mathbb{E}}^{{\mathcal{F}}_{i-1}}\left[{\Lambda }_{ki}^p\right]\le C{\varphi }_i\left(h\right)$$

Thus, we are now in a position to apply the aforementioned exponential inequality and we get: for all $\eta >0$ $$\begin{array}{c}\mathbb{P}\left(\left|{\widehat{\lambda }}_N\left(x_k,a,\widehat{s}\left(x\right)\right)-{\overline{\lambda }}_N\left(x_k,a,\widehat{s}\left(x\right)\right)\right|>\eta \sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)=\mathbb{P}\left(\frac{1}{n}\left|\sum _{i=1}^n{\Lambda }_{ki}\right|>\eta \sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)\\ \le 2\hspace{0.17em}\text{exp}\hspace{0.17em}\left\{-C{\eta }^2{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)\right\}\end{array}$$

Thus, by using the fact that ${\Gamma }_{S_{\mathcal{F}}}(ϵ)=\hspace{0.17em}\text{log}\hspace{0.17em}{N}_{ϵ}(S_{\mathcal{F}})$ and by choosing η such that $C{\eta }^2=\beta ,$ we have $$\begin{array}{c}{N}_{ϵ}\left(S_{\mathcal{F}}\right)\underset{{}_{k\in \left\{1,\dots ,N_{ϵ}\left(S_{\mathcal{F}}\right)\right\}}}{\max }\mathbb{P}\left(\left|{\widehat{\lambda }}_N\left(x_k,a,\widehat{s}\left(x\right)\right)-{\overline{\lambda }}_N\left(x_k,a,\widehat{s}\left(x\right)\right)\right|>\eta \sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)\\ \le C^{\prime }{N}_{ϵ}{\left(S_{\mathcal{F}}\right)}^{1-\beta }\end{array}$$

As ${\sum }_{n=1}^{\infty }{N}_{ϵ}{(S_{\mathcal{F}})}^{1-\beta }<\infty ,$ we obtain that (A6) $$G_2=O_{a.co.}\left(\sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)$$(A6)

Now, Lemma A.4 can be easily deduced from (A4)(A4) $$G_1=O_{a.co}\left(\sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)$$(A4) –(A6)(A6) $$G_2=O_{a.co.}\left(\sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(ϵ\right)}{n\varphi \left(h\right)}}\right)$$(A6) .

Proof of Theorem 4.3.

We use the fact that ψ_x is monotone w.r.t. the second component. We give the proof for the case of an increasing ${\psi }_x\left(\frac{Y-a}{\sigma }\right),$ decreasing case being obtained by the same manner. For this consideration, we write, $\forall ϵ>0$ $$\begin{array}{c}\sum _n\mathbb{P}\left[|\widehat{\theta }\left(x\right)-\theta \left(x\right)|\ge ϵ\right]\le \sum _n\mathbb{P}\left[\left(|\widehat{\theta }\left(x\right)-\theta \left(x\right)|1_{\left\{|\widehat{\theta }\left(x\right)-\theta \left(x\right)|\le \delta \right\}}\right)\ge ϵ\right]\\ +\sum _n\mathbb{P}\left[\left(|\widehat{\theta }\left(x\right)-\theta \left(x\right)|{\mathbb{P}}_{\left\{|\widehat{\theta }\left(x\right)-\theta \left(x\right)|>\delta \right\}}\right)\ge ϵ\right]\end{array}$$

Because ${\psi }_x\left(\frac{Y-a}{\sigma }\right)$ is increasing, it follows that, $$\begin{array}{c}\mathbb{P}\left(\left(|\widehat{\theta }\left(x\right)-\theta \left(x\right)|1_{\left\{|\widehat{\theta }\left(x\right)-\theta \left(x\right)|>{\delta }_0\right\}}\right)\ge ϵ\right)\le \mathbb{P}\left(|\widehat{\theta }\left(x\right)-\theta \left(x\right)|>{\delta }_0\right)\\ \begin{array}{cc}& \le \mathbb{P}\left(|\widehat{\lambda }\left(x,\theta \left(x\right)+\delta ,\widehat{s}\right)-\lambda \left(x,\theta \left(x\right)+\delta ,\widehat{s}\right)|\ge \lambda \left(x,\theta \left(x\right)+\delta ,\widehat{s}\right)\right)\\ & \\ & +\mathbb{P}\left(|\widehat{\lambda }\left(x,\theta \left(x\right)-\delta ,\widehat{s}\right)-\lambda \left(x,\theta \left(x\right)-\delta ,\widehat{s}\right)|\ge -\lambda \left(x,\theta \left(x\right)-\delta ,\widehat{s}\right)\right).\end{array}\end{array}$$

We have $$\left(\widehat{\theta }\left(x\right)-\theta \left(x\right)\right)1_{\left\{|\widehat{\theta }\left(x\right)-\theta \left(x\right)|\le \delta \right\}}=\frac{\lambda \left(x,\widehat{\theta }\left(x\right),\widehat{s}\right)-\widehat{\lambda }\left(x,\widehat{\theta }\left(x\right),\widehat{s}\right)}{{\lambda }^{\prime }\left(x,{\xi }_n,\widehat{s}\right)}$$ where ξ_n is between $\widehat{{\theta }_x}$ and θ_x. Therefore, all we have to prove, is to study the convergence rate of $$\underset{a\in [{\theta }_x-\delta , {\theta }_x+\delta ]}{\sup }|\lambda \left(x,a,\widehat{s}\right)-\widehat{\lambda }\left(x,a,\widehat{s}\right)|$$ and to show that (A7) $$\exists \tau >0,\sum _{n=1}^{\infty }\mathbb{P}\left({\lambda }^{\prime }\left(x,{\xi }_n,\widehat{s}\right)<\tau \right)<\infty$$(A7)

To do that, we write $$\widehat{\lambda }\left(x,a,\widehat{s}\right)=B_n\left(x,a,\widehat{s}\right)+\frac{R_n\left(x,a,\widehat{s}\right)}{{\widehat{\lambda }}_D\left(x\right)}+\frac{Q_n\left(x,a,\widehat{s}\right)}{{\widehat{\lambda }}_D\left(x\right)}$$ where $$\begin{array}{l}{Q}_n\left(x,a,\widehat{s}\right):=\left({\widehat{\lambda }}_N\left(x,a,\widehat{s}\right)-{\overline{\lambda }}_N\left(x,a,\widehat{s}\right)\right)-\lambda \left(x,a,\widehat{s}\right)\left({\widehat{\lambda }}_D\left(x\right)-{\overline{\lambda }}_D\left(x\right)\right)\\ B_n\left(x,a,\widehat{s}\right):=\frac{{\overline{\lambda }}_N\left(x,a,\widehat{s}\right)}{{\overline{\lambda }}_D\left(x\right)}-\lambda \left(x,a,\widehat{s}\right),\\ R_n\left(x,a,\widehat{s}\right):=-B_n\left(x,a,\widehat{s}\right)\left({\widehat{\lambda }}_N\left(x,a,\widehat{s}\right)-{\overline{\lambda }}_N\left(x,a,\widehat{s}\right)\right)\end{array}$$

Therefore, Theorem 4.3 is a consequence of the following intermediate results.

Lemma A.5.

Under Hypotheses $(\mathbf{H}1)$ and $(\mathbf{H}4)$, we have, $${\widehat{\lambda }}_D(x)-{\overline{\lambda }}_D(x)=O\left(\sqrt{\frac{{\Gamma }_{S_{\mathcal{F}}}\left(\hspace{0.17em}\text{log}\hspace{0.17em}\left(n\right)/n\right)}{n\varphi \left(h\right)}}\right)\hspace{1em}a.co$$

Corollary A.6.

Under Hypotheses of Lemma A.5, we have, $$\exists C>0\hspace{1em}\sum _{n=1}^{\infty }\mathbb{P}\left({\widehat{\lambda }}_D(x)<C\right)<\infty$$

Lemma A.7.

Under Hypotheses $(\mathbf{H}1),(\mathbf{H}2)$ and $(\mathbf{H}4),$ $$\underset{a\in [\theta \left(x\right)-\delta ,\theta \left(x\right)+\delta ]}{\sup }\left|B_n\left(x,a,\widehat{s}\right)\right|=O(h^{b_1})$$

Lemma A.8.

Under Assumptions $(\mathbf{H}1),(\mathbf{H}5)((\mathbf{i})-(\mathbf{ii}))$ and $(\mathbf{H}4)(\mathbf{ii}),(\mathbf{H}6),$ $\widehat{{\theta }_x}$ exists a.s. for all sufficiently large n and there exists ${\zeta }_1>0$ such that $$\sum _{n\ge 1}\mathbb{P}\left\{{\lambda }^{\prime }(x,{\xi }_n,\widehat{s})<{\zeta }_1\right\}<\infty$$

The proof of Lemma A.5 and Lemma A.7 are, respectively, very close to those of Lemma 1 and Lemma 2 in Laïb and Louani (2011). Moreover, the proof of the first part of Lemma A.8 is similar to the Lemma 5 in Azzedine, Laksaci, and Ould-Said (2008) and its second part is a direct consequence of the regularity assumption on $\lambda (x,a,\widehat{s}).$

Proof of Corollary A.6.

It is clear that, under $(\mathbf{H}2),$ there exists $0<C<C^{\prime }<\infty$ $$0<C\frac{1}{n\varphi \left(h\right)}\sum _{i=1}^n\mathbb{P}\left(X_i\in B\left(x,r\right)|{\mathcal{F}}_{i-1}\right)<{\overline{\lambda }}_D\left(x\right)<|{\widehat{\lambda }}_D\left(x\right)-{\overline{\lambda }}_D\left(x\right)|+{\widehat{\lambda }}_D\left(x\right)$$

Hence, $$\begin{array}{c}\mathbb{P}\left({\widehat{\lambda }}_D\left(x\right)\le \frac{C}{2}\right)\le \mathbb{P}\left(\frac{C}{n\varphi \left(h\right)}\sum _{i=1}^n{\mathbb{P}}^{{\mathcal{F}}_{i-1}}\left(X_i\in B\left(x,r\right)\right)<\frac{C}{2}+\left|{\widehat{\lambda }}_D\left(x\right)-{\overline{\lambda }}_D\left(x\right)\right|\right)\\ \le \mathbb{P}\left(\left|\frac{C}{n\varphi \left(h\right)}\sum _{i=1}^n{\mathbb{P}}^{{\mathcal{F}}_{i-1}}\left(X_i\in B\left(x,r\right)\right)-\left|{\widehat{\lambda }}_D\left(x\right)-{\overline{\lambda }}_D\left(x\right)\right|-C\right|>\frac{C}{2}\right)\end{array}$$

It is obvious that the previous Lemma and $(\mathbf{H}1)(\mathbf{iii})$ allows to get $\sum _n\mathbb{P}(\left|\frac{C}{n\varphi (h)}{\sum }_{i=1}^n\mathbb{P}(X_i\in B(x,r)|{\mathcal{F}}_{i-1})-\left|{\widehat{\lambda }}_D(x)-{\overline{\lambda }}_D(x)\right|-C\right|>\frac{C}{2})$ which gives the result.