Homogeneity Pursuit in Single Index Models based Panel Data Analysis

Abstract

Panel data analysis is an important topic in statistics and econometrics. Traditionally, in panel data analysis, all individuals are assumed to share the same unknown parameters, e.g. the same coefficients of covariates when the linear models are used, and the differences between the individuals are accounted for by cluster effects. This kind of modelling only makes sense if our main interest is on the global trend, this is because it would not be able to tell us anything about the individual attributes which are sometimes very important. In this paper, we propose a modelling based on the single index models embedded with homogeneity for panel data analysis, which builds the individual attributes in the model and is parsimonious at the same time. We develop a data driven approach to identify the structure of homogeneity, and estimate the unknown parameters and functions based on the identified structure. Asymptotic properties of the resulting estimators are established. Intensive simulation studies conducted in this paper also show the resulting estimators work very well when sample size is finite. Finally, the proposed modelling is applied to a public financial dataset and a UK climate dataset, the results reveal some interesting findings.

Keywords and phrases:

• Binary segmentation
• B-Spline
• homogeneity pursuit
• single index models

Acknowledgements

The authors sincerely thank the Editor Professor Christian Hansen, the Associate Editor and two anonymous reviewers for their insightful comments that significantly improve the paper. The research of Heng Lian is supported by Hong Kong RGC general research fund 11301718, and by Project 11871411 from NSFC and the Shenzhen Research Institute, City University of Hong Kong. This research is also supported by National Natural Science Foundation of China (Grant Number 71833004)

Appendix

The Appendix includes additional notations and brief technical proofs supporting Section 3 in Sections A.1 and A.2–A.5, respectively.

A.1  Additional notations

Let ${ϵ}_t={({ϵ}_{1t},\dots ,{ϵ}_{\mathit{mt}})}^{\mathrm{T}}$, $y_t={(y_{1t},\dots ,y_{\mathit{mt}})}^{\mathrm{T}},$ ${\mathbf{X}}_t={({\mathbf{X}}_{1t}^{\mathrm{T}},\dots ,{\mathbf{X}}_{\mathit{mt}}^{\mathrm{T}})}^{\mathrm{T}}.$ Due to assumption (C3), there exists ${\mathit{\theta }}_0={({\mathit{\theta }}_{01}^{\mathrm{T}},\dots ,{\mathit{\theta }}_{0m}^{\mathrm{T}})}^{\mathrm{T}},{\mathit{\theta }}_{0i}={({\theta }_{0i1},\dots ,{\theta }_{0iK})}^{\mathrm{T}}$ such that $\underset{x}{\sup }|g_{0i}(x)-{\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}(x)|\le C{K}^{-2}$. Here and below we use C to denote a generic positive constant whose value can change even on the same line. We use $\Vert .{\Vert }_{\mathit{op}}$ to denote the operator norm of a matrix (the operator norm is the same as the largest singular value) and use $\Vert .\Vert$ to denote the Frobenius norm of a matrix. We use $\Vert .{\Vert }_{L^2}$ to denote the $L^2$ norm of functions and $\Vert .{\Vert }_{\infty }$ is the sup-norm for vectors (maximum absolute value of the components).

Assume the true partition of components of ${\mathit{\theta }}_0$ and ${\overline{\mathit{\beta }}}_0$ is given by ${\cup }_{h=1}^{H_1}{G}_{1,h}=\{1,\dots ,\mathit{mK}\}$ and ${\cup }_{h=1}^{H_2}{G}_{2,h}=\{1,\dots ,\mathit{mp}\}$, respectively. The unique values of the components of ${\mathit{\theta }}_0$ and $\overline{\mathit{\beta }}$ are denoted by ${\mathbf{\xi }}_0={({\xi }_{01},\dots ,{\xi }_{0H_1})}^{\mathrm{T}}\in R^{H_1}$ and ${\mathit{\eta }}_0={({\eta }_{01},\dots ,{\eta }_{0H_2})}^{\mathrm{T}}\in R^{H_2}$, respectively. Let ${\mathbf{J}}_i^{G_1}$ be the $K\times H_1$ binary matrix whose (k, h) entry is 1 if ${\theta }_{0ik}={\xi }_h$ and 0 otherwise. We have ${\mathit{\theta }}_{0i}={\mathbf{J}}_i^{G_1}{\mathbf{\xi }}_0$. Similarly, we define ${\mathbf{J}}_i^{G_2}$ such that ${\mathit{\beta }}_{0i}={\mathbf{J}}_i^{G_2}{\mathit{\eta }}_0$. The sizes of $G_{1,h}$ and $G_{2,h}$ are denoted by $\left|G_{1,h}\right|$ and $\left|G_{2,h}\right|$. Finally, let ${\mathbf{D}}^{G_1}$ and ${\mathbf{D}}^{G_2}$ be the diagonal matrix with entries $\sqrt{\left|G_{1,h}\right|}$ and $\sqrt{\left|G_{2,h}\right|}$, respectively.

A.2  Proof summary

We first define the oracle estimator as the minimizer $(\widehat{\mathit{\theta }},\widehat{\mathit{\beta }})$ of $$\underset{\mathit{\theta },\overline{\mathit{\beta }}}{\min }\sum _{i=1}^m\sum _{t=1}^T(y_{\mathit{it}}-{\mathbf{B}}^{\mathrm{T}}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i){\mathit{\theta }}_i{)}^2,$$

where ${\mathit{\beta }}_i={(1,{\overline{\mathit{\beta }}}_i^{\mathrm{T}})}^{\mathrm{T}}={(1,{\beta }_{i1},\dots ,{\beta }_{\mathit{ip}})}^{\mathrm{T}}$ and ${\mathit{\theta }}_i={({\theta }_{i1},\dots ,{\theta }_{\mathit{iK}})}^{\mathrm{T}}$ with the constraint that components of $\overline{\mathit{\beta }}={({\overline{\mathit{\beta }}}_1^{\mathrm{T}},\dots ,{\overline{\mathit{\beta }}}_m^{\mathrm{T}})}^{\mathrm{T}}$ in the same partition take the same value and components of $\mathit{\theta }={({\mathit{\theta }}_1^{\mathrm{T}},\dots ,{\mathit{\theta }}_m^{\mathrm{T}})}^{\mathrm{T}}$ in the same partition take the same value. Here we assume the partition is the true partition, thus the name “oracle”.

Below we first show that the oracle estimator satisfies the asymptotic normality properties stated in Theorem 2 (we also obtained convergence rate and asymptotic normality for the entire vector $\mathit{\beta }$ and $\mathit{\theta }$, see for example (A.9) and (A.10). Also, noting that all arguments carry over when the partition used in the oracle estimator is finer than the true partition, Theorem 1 follows directly as a special case that each component of $\mathit{\theta }$ and $\mathit{\beta }$ forms its own group in the partition. Then we show that the change points can be consistently estimated, and thus the estimator we obtain in stage 3 will be exactly the same as the oracle estimator using the true partition, with probability approaching one, and Theorem 2 is proved. The rest of the appendix contains a sketch of the proofs outlined above while more details are relegated to the supplementary material, as well as several lemmas.

A.3  Proof of asymptotic property for the oracle estimator

In this part we consider the asymptotic property of the oracle estimator, denoted by $(\widehat{\mathit{\theta }},\widehat{\mathit{\beta }})$ in this section, which assumed knowledge of the true partitions. For clarity of presentation, the proof is split into several steps.

STEP 1. Prove the convergence rate $\Vert \widehat{\mathit{\theta }}-{\mathit{\theta }}_0\Vert +\Vert \widehat{\mathit{\beta }}-{\mathit{\beta }}_0\Vert =O_p(\sqrt{(H_1+H_2)/T}+\sqrt{m}{K}^{-2})$.

In this section, when we use $\mathit{\theta }$, we always assume ${\mathit{\theta }}_i={\mathbf{J}}_i^{G_1}\mathbf{\xi }$ for some $\mathbf{\xi }\in R^{H_1}$ (that is, components of $\mathit{\theta }$ are partitioned in the same way as is the true ${\mathit{\theta }}_0$). It is easy to see that $\Vert \mathit{\theta }-{\mathit{\theta }}_0\Vert =\Vert {\mathbf{D}}^{G_1}(\mathbf{\xi }-{\mathbf{\xi }}_0)\Vert$. Similarly, we always assume ${\mathit{\beta }}_i={\mathbf{J}}_i^{G_2}\mathit{\eta }$ for some $\mathit{\eta }\in R^{H_2}$ and $\Vert \mathit{\beta }-{\mathit{\beta }}_0\Vert =\Vert {\mathbf{D}}^{G_2}(\mathit{\eta }-{\mathit{\eta }}_0)\Vert$.

Define $r_T=\sqrt{(H_1+H_2)/T}+\sqrt{m}{K}^{-2}$. We only need to show that $$\underset{\Vert \mathit{\beta }-{\mathit{\beta }}_0{\Vert }^2+\Vert \mathit{\theta }-{\mathit{\theta }}_0{\Vert }^2=L{r}_T^2}{\inf }\sum _{i=1}^m\sum _{t=1}^T(y_{\mathit{it}}-{\mathit{\theta }}_i^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i){)}^2-\sum _{i=1}^m\sum _{t=1}^T(y_{\mathit{it}}-{\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}){)}^2>0$$

with probability approaching one, if L is large enough.

We have $$\begin{array}{c}\sum _{i=1}^m\sum _{t=1}^T(y_{\mathit{it}}-{\mathit{\theta }}_i^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i){)}^2-\sum _{i=1}^m\sum _{t=1}^T(y_{\mathit{it}}-{\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}){)}^2\\ =\sum _{i,t}({\mathit{\theta }}_i^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i)-{\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}){)}^2-2({ϵ}_{\mathit{it}}-r_{\mathit{it}})({\mathit{\theta }}_i^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i)-{\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i})),\end{array}$$

where $r_{\mathit{it}}={\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i})-g({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i})$ with $\left|r_{\mathit{it}}\right|\le C{K}^{-2}$.

Furthermore, some algebra shows $$\begin{array}{c}\sum _{i,t}({\mathit{\theta }}_i^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i)-{\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}){)}^2\\ =T(({\mathit{\xi }}^{\mathrm{T}}-{\mathit{\xi }}_0^{\mathrm{T}}){\mathbf{D}}^{G_1},({\mathit{\eta }}^{\mathrm{T}}-{\mathit{\eta }}_0^{\mathrm{T}}){\mathbf{D}}^{G_2})\\ \left(\begin{array}{cc}{({\mathbf{D}}^{G_1})}^{-1}& 0\\ 0& {({\mathbf{D}}^{G_2})}^{-1}\end{array}\right)\left(\begin{array}{ccccc}{({\mathbf{J}}_1^{G_1})}^{\mathrm{T}}& 0& \cdots & {({\mathbf{J}}_m^{G_1})}^{\mathrm{T}}& 0\\ 0& {({\mathbf{J}}_1^{G_2})}^{\mathrm{T}}& \cdots & 0& {({\mathbf{J}}_m^{G_2})}^{\mathrm{T}}\end{array}\right)\\ \left(\begin{array}{cccc}{\tilde{\mathbf{A}}}_1& 0& \cdots & 0\\ 0& {\tilde{\mathbf{A}}}_2& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮\\ 0& 0& \cdots & {\tilde{\mathbf{A}}}_m\end{array}\right)·\left(\begin{array}{cc}{\mathbf{J}}_1^{G_1}& 0\\ 0& {\mathbf{J}}_1^{G_2}\\ ⋮& ⋮\\ {\mathbf{J}}_m^{G_1}& 0\\ 0& {\mathbf{J}}_m^{G_2}\end{array}\right)\left(\begin{array}{cc}{({\mathbf{D}}^{G_1})}^{-1}& 0\\ 0& {({\mathbf{D}}^{G_2})}^{-1}\end{array}\right)\left(\begin{array}{c}{\mathbf{D}}^{G_1}(\mathit{\xi }-{\mathit{\xi }}_0)\\ {\mathbf{D}}^{G_2}(\mathit{\eta }-{\mathit{\eta }}_0)\end{array}\right),\end{array}$$

where $${\tilde{\mathbf{A}}}_i=\frac{1}{T}\sum _{t=1}^T\left[\left(\begin{array}{c}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i)\\ {\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}\prime ({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i^{*}){\overline{\mathbf{X}}}_{\mathit{it}}\end{array}\right)\left(\begin{array}{cc}{\mathbf{B}}^{\mathrm{T}}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i)& {\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}\prime ({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i^{*}){\overline{\mathbf{X}}}_{\mathit{it}}^{\mathrm{T}}\end{array}\right)\right],1\le i\le m,$$

$\mathbf{B}\prime (.)={(B_1^{\prime }(.),\dots ,B_K^{\prime }(.))}^{\mathrm{T}}$ are the first derivatives of the basis functions and ${\mathit{\beta }}_i^{*}$ lies between ${\mathit{\beta }}_{0i}$ and ${\mathit{\beta }}_i$.

With the help of Lemma 3 in the SUPPLEMENTARY MATERIAL provided in a separated file, and noting that (A.1) $$\mathbf{O}:=\left(\begin{array}{cc}{\mathbf{J}}_1^{G_1}& 0\\ 0& {\mathbf{J}}_1^{G_2}\\ ⋮& ⋮\\ {\mathbf{J}}_m^{G_1}& 0\\ 0& {\mathbf{J}}_m^{G_2}\end{array}\right)\left(\begin{array}{cc}{({\mathbf{D}}^{G_1})}^{-1}& 0\\ 0& {({\mathbf{D}}^{G_2})}^{-1}\end{array}\right)$$(A.1)

is an orthonormal matrix (that is, ${\mathbf{O}}^{\mathrm{T}}\mathbf{O}=\mathbf{I}$), we can get (A.2) $$\sum _{i,t}({\mathit{\theta }}_i^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i)-{\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}){)}^2\asymp T(\Vert {\mathbf{D}}^{G_1}(\mathit{\xi }-{\mathit{\xi }}_0){\Vert }^2+\Vert {\mathbf{D}}^{G_2}(\mathit{\eta }-{\mathit{\eta }}_0){\Vert }^2)=T(\Vert \mathit{\theta }-{\mathit{\theta }}_0{\Vert }^2+\Vert \mathit{\beta }-{\mathit{\beta }}_0{\Vert }^2).$$(A.2)

Now consider the term $({ϵ}_{\mathit{it}}-r_{\mathit{it}})({\mathit{\theta }}_i^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i)-{\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}))$. We can show $$\begin{array}{c}\sum _{i,t}{ϵ}_{\mathit{it}}({\mathit{\theta }}_i^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i)-{\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}))\\ \le \sqrt{\Vert {\mathbf{D}}^{G_1}(\mathit{\xi }-{\mathit{\xi }}_0){\Vert }^2+\Vert {\mathbf{D}}^{G_2}(\mathit{\eta }-{\mathit{\eta }}_0){\Vert }^2}\Vert \sum _t{\mathbf{O}}^{\mathrm{T}}\left(\begin{array}{c}\mathbf{B}({\mathbf{X}}_{1t}^{\mathrm{T}}{\mathit{\beta }}_1){ϵ}_{1t}\\ {\mathit{\theta }}_{01}^{\mathrm{T}}\mathbf{B}\prime ({\mathbf{X}}_{1t}^{\mathrm{T}}{\mathit{\beta }}_1^{*}){\overline{\mathbf{X}}}_{1t}{ϵ}_{1t}\\ ⋮\\ \mathbf{B}({\mathbf{X}}_{\mathit{mt}}^{\mathrm{T}}{\mathit{\beta }}_m){ϵ}_{\mathit{mt}}\\ {\mathit{\theta }}_{0m}^{\mathrm{T}}\mathbf{B}\prime ({\mathbf{X}}_{\mathit{mt}}^{\mathrm{T}}{\mathit{\beta }}_m^{*}){\overline{\mathbf{X}}}_{\mathit{mt}}{ϵ}_{\mathit{mt}}\end{array}\right)\Vert ,\end{array}$$

where O is as defined in (A.1), and further calculations reveal that (A.3) $$\begin{array}{c}E{\Vert \sum _t{\mathbf{O}}^{\mathrm{T}}\left(\begin{array}{c}\mathbf{B}({\mathbf{X}}_{1t}^{\mathrm{T}}{\mathit{\beta }}_1){ϵ}_{1t}\\ {\mathit{\theta }}_{01}^{\mathrm{T}}\mathbf{B}\prime ({\mathbf{X}}_{1t}^{\mathrm{T}}{\mathit{\beta }}_1^{*}){\overline{\mathbf{X}}}_{1t}{ϵ}_{1t}\\ ⋮\\ \mathbf{B}({\mathbf{X}}_{\mathit{mt}}^{\mathrm{T}}{\mathit{\beta }}_m){ϵ}_{\mathit{mt}}\\ {\mathit{\theta }}_{0m}^{\mathrm{T}}\mathbf{B}\prime ({\mathbf{X}}_{\mathit{mt}}^{\mathrm{T}}{\mathit{\beta }}_m^{*}){\overline{\mathbf{X}}}_{\mathit{mt}}{ϵ}_{\mathit{mt}}\end{array}\right)\Vert }^2\\ \le \text{tr}(\mathbf{O}{\mathbf{O}}^{\mathrm{T}})·{\Vert \sum _{1\le t,t\prime \le T}\left[\begin{array}{cccc}{\mathbf{A}}_{1,|t-t\prime |}& 0& \cdots & 0\\ 0& {\mathbf{A}}_{2,|t-t\prime |}& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮\\ 0& 0& \cdots & {\mathbf{A}}_{m,|t-t\prime |}\end{array}\right]\Vert }_{\mathit{op}},\end{array}$$(A.3)

where $${\mathbf{A}}_{i,|t-t\prime |}=E\left[\left(\begin{array}{c}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i)\\ {\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}\prime ({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i^{*}){\overline{\mathbf{X}}}_{\mathit{it}}\end{array}\right)\left(\begin{array}{cc}{\mathbf{B}}^{\mathrm{T}}({\mathbf{X}}_{\mathit{it}\prime }^{\mathrm{T}}{\mathit{\beta }}_i)& {\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}\prime ({\mathbf{X}}_{\mathit{it}\prime }^{\mathrm{T}}{\mathit{\beta }}_i^{*}){\overline{\mathbf{X}}}_{\mathit{it}\prime }^{\mathrm{T}}\end{array}\right){ϵ}_{\mathit{it}}{ϵ}_{\mathit{it}\prime }\right].$$

By Lemma 4 in the SUPPLEMENTARY MATERIAL provided in a separated file, and that $\text{tr}(\mathbf{O}{\mathbf{O}}^{\mathrm{T}})=H_1+H_2$ (note ${\mathbf{O}}^{\mathrm{T}}\mathbf{O}={\mathbf{I}}_{H_1+H_2}$), we have (A.4) $$\begin{array}{c}\sum _{i,t}{ϵ}_{\mathit{it}}({\mathit{\theta }}_i^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i)-{\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}))\\ =O_p(\sqrt{(\Vert \mathit{\theta }-\mathit{\theta }{\Vert }^2+\Vert \mathit{\beta }-{\mathit{\beta }}_0{\Vert }^2)(H_1+H_2)T}).\end{array}$$(A.4)

Finally, using Cauchy-Schwarz inequality (A.5) $$\begin{array}{c}\sum _{i,t}{r}_{\mathit{it}}({\mathit{\theta }}_i^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i)-{\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}))\\ =C\sqrt{\mathit{mT}}{K}^{-2}·O_p(\sqrt{T(\Vert \mathit{\theta }-\mathit{\theta }{\Vert }^2+\Vert \mathit{\beta }-{\mathit{\beta }}_0{\Vert }^2)})\end{array}$$(A.5)

Combining (A.2)–(A.5), $$\sum _{i=1}^m\sum _{t=1}^T(y_{\mathit{it}}-{\mathit{\theta }}_i^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i){)}^2-\sum _{i=1}^m\sum _{t=1}^T(y_{\mathit{it}}-{\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}){)}^2>0$$

with probability approaching one, if $\Vert \mathit{\beta }-{\mathit{\beta }}_0{\Vert }^2+\Vert \mathit{\theta }-{\mathit{\theta }}_0{\Vert }^2=L{r}_T^2$ with L sufficiently large. Thus there is a local minimizer $(\widehat{\mathit{\theta }},\widehat{\mathit{\beta }})$ with $\Vert \widehat{\mathit{\beta }}-{\mathit{\beta }}_0\Vert +\Vert \widehat{\mathit{\theta }}-{\mathit{\theta }}_0\Vert =O_p(r_T)$.

STEP 2. Proof of convergence rate of $\widehat{\mathit{\beta }}$ and its asymptotic normality.

Let ${\mathbf{\Pi }}_i$ be $T\times K$ matrices, $i=1,\dots ,m$, with rows ${\mathbf{\Pi }}_{\mathit{it}}^{\mathrm{T}}={\mathbf{B}}^{\mathrm{T}}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i})$. Define ${\mathbf{V}}_{\mathit{it}}=g_{0i}^{\prime }({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}){\overline{\mathbf{X}}}_{\mathit{it}},{\mathbf{P}}_i={\mathbf{\Pi }}_i{({\mathbf{\Pi }}_i^{\mathrm{T}}{\mathbf{\Pi }}_i)}^{-1}{\mathbf{\Pi }}_i^{\mathrm{T}}$ with rows ${\mathbf{P}}_{\mathit{it}}^{\mathrm{T}}={\mathbf{\Pi }}_{\mathit{it}}^{\mathrm{T}}{({\mathbf{\Pi }}_i^{\mathrm{T}}{\mathbf{\Pi }}_i)}^{-1}{\mathbf{\Pi }}_i^{\mathrm{T}}$. We write, for any $(\mathit{\theta },\mathit{\beta })$ with $\Vert \mathit{\theta }-{\mathit{\theta }}_0{\Vert }^2\le C{r}_T^2$ and $\Vert \mathit{\beta }-{\mathit{\beta }}_0{\Vert }^2\le C{H}_2/T$, $$\begin{array}{c}\sum _{i,t}(y_{\mathit{it}}-{\mathit{\theta }}_i^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i){)}^2\\ =\sum _{i,t}({ϵ}_{\mathit{it}}-{\mathbf{\Pi }}_{\mathit{it}}^{\mathrm{T}}({\mathit{\theta }}_i-{\mathit{\theta }}_{0i})-{\mathbf{V}}_{\mathit{it}}^{\mathrm{T}}({\overline{\mathit{\beta }}}_i-{\overline{\mathit{\beta }}}_{0i})-R_{\mathit{it}}{)}^2,\end{array}$$

where $$\begin{array}{c}{R}_{\mathit{it}}\\ =\left\{{\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i})-g_{0i}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i})\right\}+\left\{{({\mathit{\theta }}_i-{\mathit{\theta }}_{0i})}^{\mathrm{T}}(\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i)-\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}))\right\}\\ +\left\{{\mathit{\theta }}_{0i}^{\mathrm{T}}(\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i)-\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i})-\mathbf{B}\prime ({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}){\overline{\mathbf{X}}}_{\mathit{it}}^{\mathrm{T}}({\overline{\mathit{\beta }}}_i-{\overline{\mathit{\beta }}}_{0i})\right\}\\ +\left\{({\mathit{\theta }}_{0i}^{\mathrm{T}}\mathbf{B}\prime ({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i})-g_{0i}^{\prime }({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i})){\overline{\mathbf{X}}}_{\mathit{it}}^{\mathrm{T}}({\overline{\mathit{\beta }}}_i-{\overline{\mathit{\beta }}}_{0i})\right\}\\ =R_{\mathit{it}1}+R_{\mathit{it}2}({\mathit{\theta }}_i,{\mathit{\beta }}_i),\end{array}$$

with $R_{\mathit{it}1}={\mathit{\theta }}_0^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i})-g_{0i}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i})$ and $R_{\mathit{it}2}({\mathit{\theta }}_i,{\mathit{\beta }}_i)$ (or $R_{\mathit{it}2}$ for short) contains the remaining terms. Using $\Vert \mathit{\theta }-{\mathit{\theta }}_0{\Vert }^2+\Vert \mathit{\beta }-{\mathit{\beta }}_0{\Vert }^2\le C{r}_T^2$, we can show (A.6) $$\sum _{i,t}{R}_{\mathit{it}2}^2=O_p\left(T{r}_T^4{K}^3+T{r}_T^2{K}^{-2}\right).$$(A.6)

We then orthogonalize the parametric part with respect to the nonparametric part by writing $$\begin{array}{c}\sum _{i,t}({ϵ}_{\mathit{it}}-{\mathbf{\Pi }}_{\mathit{it}}^{\mathrm{T}}({\mathit{\theta }}_i-{\mathit{\theta }}_{0i})-{\mathbf{V}}_{\mathit{it}}^{\mathrm{T}}({\overline{\mathit{\beta }}}_i-{\overline{\mathit{\beta }}}_{0i})-R_{\mathit{it}}{)}^2\\ =\sum _{i,t}({ϵ}_{\mathit{it}}-{\mathbf{\Pi }}_{\mathit{it}}^{\mathrm{T}}({\mathit{\alpha }}_i-{\mathit{\alpha }}_{0i})-{({\mathbf{V}}_{\mathit{it}}-{\mathbf{V}}_i^{\mathrm{T}}{\mathbf{P}}_{\mathit{it}})}^{\mathrm{T}}({\overline{\mathit{\beta }}}_i-{\overline{\mathit{\beta }}}_{0i})-R_{\mathit{it}1}-R_{\mathit{it}2}{({\mathcal{M}}_i({\mathit{\alpha }}_i,{\mathit{\beta }}_i))}^2,\end{array}$$

where ${\mathit{\alpha }}_i={\mathit{\theta }}_i+{({\mathbf{\Pi }}_i^{\mathrm{T}}{\mathbf{\Pi }}_i)}^{-1}{\mathbf{\Pi }}_i^{\mathrm{T}}{\mathbf{V}}_i{\overline{\mathit{\beta }}}_i,{\mathit{\alpha }}_{0i}={\mathit{\theta }}_{0i}+{({\mathbf{\Pi }}_i^{\mathrm{T}}{\mathbf{\Pi }}_i)}^{-1}{\mathbf{\Pi }}_i^{\mathrm{T}}{\mathbf{V}}_i{\overline{\mathit{\beta }}}_{0i},{\mathbf{V}}_i={({\mathbf{V}}_{\mathit{it}},\dots ,{\mathbf{V}}_{\mathit{iT}})}^{\mathrm{T}}$, and ${\mathcal{M}}_i$ is the one-to-one mapping that maps $({\mathit{\alpha }}_i,{\mathit{\beta }}_i)$ to $({\mathit{\theta }}_i,{\mathit{\beta }}_i)$. Below we write $R_{\mathit{it}2}({\mathcal{M}}_i({\mathit{\alpha }}_i,{\mathit{\beta }}_i))$ as $R_{\mathit{it}2},R_{\mathit{it}2}({\mathcal{M}}_i({\widehat{\mathit{\alpha }}}_i,{\widehat{\mathit{\beta }}}_i))$ as ${\widehat{R}}_{\mathit{it}2}$, and note $R_{\mathit{it}2}({\mathcal{M}}_i({\widehat{\mathit{\alpha }}}_i,{\mathit{\beta }}_0))=0$. Then, (A.7) $$\begin{array}{c}0\ge \sum _{i,t}({ϵ}_{\mathit{it}}-{\mathbf{\Pi }}_{\mathit{it}}^{\mathrm{T}}{\widehat{\mathit{\alpha }}}_i-{({\mathbf{V}}_{\mathit{it}}-{\mathbf{V}}_i^{\mathrm{T}}{\mathbf{P}}_{\mathit{it}})}^{\mathrm{T}}({\widehat{\overline{\mathit{\beta }}}}_i-{\overline{\mathit{\beta }}}_{0i})-R_{\mathit{it}1}-R_{\mathit{it}2}{)}^2\\ -\sum _{i,t}{({ϵ}_{\mathit{it}}-{\mathbf{\Pi }}_{\mathit{it}}^{\mathrm{T}}{\widehat{\mathit{\alpha }}}_i-R_{\mathit{it}1})}^2\\ =\sum _{i,t}{(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0)}^{\mathrm{T}}{({\mathbf{J}}_i^{G_2})}^{\mathrm{T}}({\mathbf{V}}_{\mathit{it}}-{\mathbf{V}}_i^{\mathrm{T}}{\mathbf{P}}_{\mathit{it}})({\mathbf{V}}_{\mathit{it}}^{\mathrm{T}}-{\mathbf{P}}_{\mathit{it}}^{\mathrm{T}}{\mathbf{V}}_i){\mathbf{J}}_i^{G_2}(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0)\\ -2\sum _{i,t}{(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0)}^{\mathrm{T}}{({\mathbf{J}}_i^{G_2})}^{\mathrm{T}}({\mathbf{V}}_{\mathit{it}}-{\mathbf{V}}_i^{\mathrm{T}}{\mathbf{P}}_{\mathit{it}}){ϵ}_{\mathit{it}}\\ -2\sum _{i,t}{\widehat{R}}_{\mathit{it}2}{ϵ}_{\mathit{it}}\\ +\sum _{i,t}{\widehat{R}}_{\mathit{it}2}^2+2\sum _{i,t}\left({(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0)}^{\mathrm{T}}{({\mathbf{J}}_i^{G_2})}^{\mathrm{T}}({\mathbf{V}}_{\mathit{it}}-{\mathbf{V}}_i^{\mathrm{T}}{\mathbf{P}}_{\mathit{it}})+{\widehat{R}}_{\mathit{it}2}\right)({\mathbf{\Pi }}_{\mathit{it}}^{\mathrm{T}}{\widehat{\mathit{\alpha }}}_i+R_{\mathit{it}1})\\ +2\sum _{i,t}{\widehat{R}}_{\mathit{it}2}·{(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0)}^{\mathrm{T}}{({\mathbf{J}}_i^{G_2})}^{\mathrm{T}}({\mathbf{V}}_{\mathit{it}}-{\mathbf{V}}_i^{\mathrm{T}}{\mathbf{P}}_{\mathit{it}}).\end{array}$$(A.7)

The first term above is $$\begin{array}{c}\sum _{i,t}(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0){({\mathbf{J}}_i^{G_2})}^{\mathrm{T}}({\mathbf{V}}_{\mathit{it}}-{\mathbf{V}}_i^{\mathrm{T}}{\mathbf{P}}_{\mathit{it}})({\mathbf{V}}_{\mathit{it}}^{\mathrm{T}}-{\mathbf{P}}_{\mathit{it}}^{\mathrm{T}}{\mathbf{V}}_i){\mathbf{J}}_i^{G_2}(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0)\\ =T{(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0)}^{\mathrm{T}}{\mathbf{D}}^{G_2}{\mathbf{O}}_2^{\mathrm{T}}\left(\begin{array}{cccc}{\widehat{\mathbf{C}}}_1& 0& \cdots & 0\\ 0& {\widehat{\mathbf{C}}}_2& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮\\ 0& 0& \cdots & {\widehat{\mathbf{C}}}_m\end{array}\right){\mathbf{O}}_2{\mathbf{D}}^{G_2}(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0),\end{array}$$

where ${\mathbf{O}}_2=(\begin{array}{c}{\mathbf{J}}_1^{G_2}\\ ⋮\\ {\mathbf{J}}_m^{G_2}\end{array}){({\mathbf{D}}^{G_2})}^{-1}$ is an $\mathit{mp}\times H_2$ orthonormal matrix, and ${\widehat{\mathbf{C}}}_i=\sum _t({\mathbf{V}}_{\mathit{it}}-{\mathbf{V}}_i^{\mathrm{T}}{\mathbf{P}}_{\mathit{it}}){({\mathbf{V}}_{\mathit{it}}-{\mathbf{V}}_i^{\mathrm{T}}{\mathbf{P}}_{\mathit{it}})}^{\mathrm{T}}/T$.

Let ${\mathbf{C}}_i=E[{(g_{0i}^{\prime }({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}))}^2{({\overline{\mathbf{X}}}_{\mathit{it}}-E[{\overline{\mathbf{X}}}_{\mathit{it}}|{\mathbf{X}}_{\mathit{it}}{\mathit{\beta }}_{0i}])}^{\otimes 2}]$. Lemma 5 in the SUPPLEMENTARY MATERIAL, provided in a separated file, shows that ${\max }_i\Vert {\widehat{\mathbf{C}}}_i-{\mathbf{C}}_i{\Vert }_{\mathit{op}}=o_p(1)$. Based on this, we have the first term in (A.7) is bounded below by $\mathit{CT}\Vert {\mathbf{D}}^{G_2}(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0){\Vert }^2$.

Now consider the second term in (A.7). We have $$\begin{array}{c}\sum _{i,t}(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0){({\mathbf{J}}_i^{G_2})}^{\mathrm{T}}({\mathbf{V}}_{\mathit{it}}-{\mathbf{V}}_i^{\mathrm{T}}{\mathbf{P}}_{\mathit{it}}){ϵ}_{\mathit{it}}\\ \le \Vert {\mathbf{D}}^{G_2}(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0)\Vert ·\Vert \sum _t{\mathbf{O}}_2^{\mathrm{T}}\left(\begin{array}{c}({\mathbf{V}}_{1t}-{\mathbf{V}}_1^{\mathrm{T}}{\mathbf{P}}_{1t}){ϵ}_{1t}\\ ⋮\\ ({\mathbf{V}}_{\mathit{mt}}-{\mathbf{V}}_m^{\mathrm{T}}{\mathbf{P}}_{\mathit{mt}}){ϵ}_{\mathit{mt}}\end{array}\right)\Vert ,\end{array}$$

and with some more detailed analyses we get (A.3), we get (A.8) $$E\left[{\Vert \sum _t{\mathbf{O}}_2^{\mathrm{T}}\left(\begin{array}{c}({\mathbf{V}}_{1t}-{\mathbf{V}}_1^{\mathrm{T}}{\mathbf{P}}_{1t}){ϵ}_{1t}\\ ⋮\\ ({\mathbf{V}}_{\mathit{mt}}-{\mathbf{V}}_m^{\mathrm{T}}{\mathbf{P}}_{\mathit{mt}}){ϵ}_{\mathit{mt}}\end{array}\right)\Vert }^2\right]=O(H_{2}T)$$(A.8)

and thus the second term in (A.7) is $O_p(\sqrt{H_{2}T}\Vert {\mathbf{D}}^{G_2}(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0)\Vert )$. The remaining terms in (A.7) can be shown to be of order $o_p(1)$. Summarizing the bounds for different terms in (A.7), we get $$\Vert {\mathbf{D}}^{G_2}(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0){\Vert }^2+\Vert {\mathbf{D}}^{G_2}(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0)\Vert O_p\left(\sqrt{H_2/T}\right)+o_p(1/T)\le 0,$$

which implies $\Vert \widehat{\mathit{\beta }}-{\mathit{\beta }}_0\Vert =\Vert {\mathbf{D}}^{G_2}(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0)\Vert =O_p\left(\sqrt{H_2/T}\right)$.

To get asymptotic normality, we define $$\tilde{\mathit{\eta }}={\mathit{\eta }}_0+{\left(\sum _{i,t}{({\mathbf{J}}_i^{G_2})}^{\mathrm{T}}({\mathbf{V}}_{\mathit{it}}-{\mathbf{V}}_i^{\mathrm{T}}{\mathbf{P}}_{\mathit{it}})({\mathbf{V}}_{\mathit{it}}^{\mathrm{T}}-{\mathbf{P}}_{\mathit{it}}^{\mathrm{T}}{\mathbf{V}}_i){\mathbf{J}}_i^{G_2}\right)}^{-1}\sum _{i,t}{({\mathbf{J}}_i^{G_2})}^{\mathrm{T}}({\mathbf{V}}_{\mathit{it}}-{\mathbf{V}}_i^{\mathrm{T}}{\mathbf{P}}_{\mathit{it}}){ϵ}_{\mathit{it}}.$$

Then for any unit vector ${\mathbf{a}}_2\in R^{H_2}$, we have $$\begin{array}{c}{\mathbf{a}}_2^{\mathrm{T}}{\mathbf{D}}^{G_2}(\tilde{\mathit{\eta }}-{\mathit{\eta }}_0)\\ =T^{-1}{\mathbf{a}}_2^{\mathrm{T}}{\left({\mathbf{O}}_2^{\mathrm{T}}\left(\begin{array}{cccc}{\widehat{\mathbf{C}}}_1& 0& \cdots & 0\\ 0& {\widehat{\mathbf{C}}}_2& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮\\ 0& 0& \cdots & {\widehat{\mathbf{C}}}_m\end{array}\right){\mathbf{O}}_2\right)}^{-1}·\\ \sum _t{\mathbf{O}}_2^{\mathrm{T}}\left(\begin{array}{c}({\mathbf{V}}_{1t}-{\mathbf{V}}_1^{\mathrm{T}}{\mathbf{P}}_{1t}){ϵ}_{1t}\\ ⋮\\ ({\mathbf{V}}_{\mathit{mt}}-{\mathbf{V}}_m^{\mathrm{T}}{\mathbf{P}}_{\mathit{mt}}){ϵ}_{\mathit{mt}}\end{array}\right).\end{array}$$

Consider $$\begin{array}{c}{b}_2:=T^{-1}{\mathbf{a}}_2^{\mathrm{T}}{\left({\mathbf{O}}_2^{\mathrm{T}}\left(\begin{array}{cccc}{\mathbf{C}}_1& 0& \cdots & 0\\ 0& {\mathbf{C}}_2& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮\\ 0& 0& \cdots & {\mathbf{C}}_m\end{array}\right){\mathbf{O}}_2\right)}^{-1}·\\ \sum _t{\mathbf{O}}_2^{\mathrm{T}}\left(\begin{array}{c}({\mathbf{V}}_{1t}-{\mathbf{\Phi }}_{1t}){ϵ}_{1t}\\ ⋮\\ ({\mathbf{V}}_{\mathit{mt}}-{\mathbf{\Phi }}_{\mathit{mt}}){ϵ}_{\mathit{mt}}\end{array}\right).\end{array}$$

We can show using the central limit theorem under mixing conditions, for example results in Bardet et al. (2008), that $$\sqrt{T}{\nu }_{2,T}^{-1/2}{b}_2\stackrel{d}{\to }N(0,1),$$

where $$\begin{array}{c}{\nu }_{2,T}={\mathbf{a}}_2^{\mathrm{T}}{({\mathbf{O}}_2^{\mathrm{T}}\mathbf{C}{\mathbf{O}}_2)}^{-1}{\mathbf{O}}_2^{\mathrm{T}}{\mathbf{\Sigma }}_2{\mathbf{O}}_2{({\mathbf{O}}_2^{\mathrm{T}}\mathbf{C}{\mathbf{O}}_2)}^{-1}{\mathbf{a}}_2,\\ \mathbf{C}=\left(\begin{array}{cccc}{\mathbf{C}}_1& 0& \cdots & 0\\ 0& {\mathbf{C}}_2& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮\\ 0& 0& \cdots & {\mathbf{C}}_m\end{array}\right),\\ {\mathbf{\Sigma }}_2=\frac{1}{T}\sum _{1\le t,t\prime \le T}\left[\begin{array}{cccc}{\mathbf{C}}_{1,|t-t\prime |}& 0& \cdots & 0\\ 0& {\mathbf{C}}_{2,|t-t\prime |}& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮\\ 0& 0& \cdots & {\mathbf{C}}_{m,|t-t\prime |}\end{array}\right],\end{array}$$

where ${\mathbf{C}}_{i,|t-t\prime |}=E[{ϵ}_{\mathit{it}}{ϵ}_{\mathit{it}\prime }{g}_{0i}^{\prime }({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i})g_{0i}^{\prime }({\mathbf{X}}_{\mathit{it}\prime }^{\mathrm{T}}{\mathit{\beta }}_{0i})({\overline{\mathbf{X}}}_{\mathit{it}}-E[{\overline{\mathbf{X}}}_{\mathit{it}}|{\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}]){({\overline{\mathbf{X}}}_{\mathit{it}\prime }-E[{\overline{\mathbf{X}}}_{\mathit{it}\prime }|{\mathbf{X}}_{\mathit{it}\prime }^{\mathrm{T}}{\mathit{\beta }}_{0i}])}^{\mathrm{T}}]$. Furthermore, it can be shown that $$|{\mathbf{a}}_2^{\mathrm{T}}{\mathbf{D}}^{G_2}(\tilde{\mathit{\eta }}-{\mathit{\eta }}_0)-b_1|=o_p(1/\sqrt{T}),$$

and $$|{\mathbf{a}}_2^{\mathrm{T}}{\mathbf{D}}^{G_2}(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0)-{\mathbf{a}}_2^{\mathrm{T}}{\mathbf{D}}^{G_2}(\tilde{\mathit{\eta }}-{\mathit{\eta }}_0)|=o_p(1/\sqrt{T}),$$

which established the asymptotic normality of $\widehat{\mathit{\eta }}$.

Since $\widehat{\overline{\mathit{\beta }}}-{\overline{\mathit{\beta }}}_0={\mathbf{O}}_2{\mathbf{D}}^{G_2}(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0)$, ${\mathbf{b}}_2^{\mathrm{T}}(\widehat{\overline{\mathit{\beta }}}-{\overline{\mathit{\beta }}}_0)={\mathbf{b}}_2^{\mathrm{T}}{\mathbf{O}}_2{\mathbf{D}}^{G_2}(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0)$ is asymptotically normal. That is, for any unit vector ${\mathbf{b}}_2\in R^{\mathit{mp}}$, (A.9) $$\sqrt{T}{\kappa }_{2,T}^{-1/2}{\mathbf{b}}_2^{\mathrm{T}}(\widehat{\overline{\mathit{\beta }}}-{\overline{\mathit{\beta }}}_0)\stackrel{d}{\to }N(0,1),$$(A.9)

where $${\kappa }_{2,T}:={\mathbf{b}}_2^{\mathrm{T}}{\mathbf{O}}_2{({\mathbf{O}}_2^{\mathrm{T}}\mathbf{C}{\mathbf{O}}_2)}^{-1}{\mathbf{O}}_2^{\mathrm{T}}{\mathbf{\Sigma }}_2{\mathbf{O}}_2{({\mathbf{O}}_2^{\mathrm{T}}\mathbf{C}{\mathbf{O}}_2)}^{-1}{\mathbf{O}}_2^{\mathrm{T}}{\mathbf{b}}_2.$$

STEP 3. Proof of the convergence rate of $\widehat{\mathit{\theta }}$ and its asymptotic normality.

To get the convergence rate of $\widehat{\mathit{\theta }}$, like for $\widehat{\mathit{\beta }}$, we perform a projection, which is now the projection for the nonparametric part. Let ${\mathbf{A}}_{0i}:=\arg {\min }_{\mathbf{A}}\Vert \mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i})-g{\prime }_{0i}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i})\mathbf{A}{\overline{\mathbf{X}}}_{\mathit{it}}{\Vert }^2$. Obviously, we have $${\mathbf{A}}_{0i}=E\left[g{\prime }_{0i}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i})\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}){\overline{\mathbf{X}}}_{\mathit{it}}^{\mathrm{T}}\right]{\left(E\left[{(g{\prime }_{0i}({\overline{\mathbf{X}}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}))}^2{\overline{\mathbf{X}}}_{\mathit{it}}{\overline{\mathbf{X}}}_{\mathit{it}}^{\mathrm{T}}\right]\right)}^{-1}.$$

Writing now that $$\begin{array}{c}\sum _{i,t}(y_{\mathit{it}}-{\mathit{\theta }}_i^{\mathrm{T}}\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_i){)}^2\\ =\sum _{i,t}({ϵ}_{\mathit{it}}-({\mathbf{\Pi }}_{\mathit{it}}^{\mathrm{T}}-{\mathbf{Q}}_{\mathit{it}}^{\mathrm{T}}{\mathbf{\Pi }}_i)({\mathit{\theta }}_i-{\mathit{\theta }}_{0i})-{\mathbf{V}}_{\mathit{it}}^{\mathrm{T}}({\mathit{\gamma }}_i-{\mathit{\gamma }}_{0i})-R_{\mathit{it}1}-R_{\mathit{it}2}({\mathit{\theta }}_i,{\mathit{\gamma }}_i-{({\mathbf{V}}_i^{\mathrm{T}}{\mathbf{V}}_i)}^{-1}{\mathbf{V}}_i^{\mathrm{T}}{\mathit{\theta }}_i){)}^2,\end{array}$$

where ${\mathbf{Q}}_{\mathit{it}}^{\mathrm{T}}$ is the t-th row of ${\mathbf{Q}}_i={\mathbf{V}}_i{({\mathbf{V}}_i^{\mathrm{T}}{\mathbf{V}}_i)}^{-1}{\mathbf{V}}_i^{\mathrm{T}}$ and ${\mathit{\gamma }}_i={\overline{\mathit{\beta }}}_i+{({\mathbf{V}}_i^{\mathrm{T}}{\mathbf{V}}_i)}^{-1}{\mathbf{V}}_i^{\mathrm{T}}{\mathbf{\Pi }}_i{\mathit{\theta }}_i,{\mathit{\gamma }}_{0i}={\overline{\mathit{\beta }}}_{0i}+{({\mathbf{V}}_i^{\mathrm{T}}{\mathbf{V}}_i)}^{-1}{\mathbf{V}}_i^{\mathrm{T}}{\mathbf{\Pi }}_i{\mathit{\theta }}_{0i}$, we can show $\Vert \widehat{\mathit{\theta }}-{\mathit{\theta }}_0{\Vert }^2=O_p(H_1/T)$ and its asymptotic normality similar to arguments used for $\widehat{\mathit{\beta }}$. In particular, we have that for any unit vector ${\mathbf{b}}_1\in R^{\mathit{mp}}$, (A.10) $$\sqrt{T}{\kappa }_{1,T}^{-1/2}{\mathbf{b}}_1^{\mathrm{T}}(\widehat{\mathit{\theta }}-{\mathit{\theta }}_0)\stackrel{d}{\to }N(0,1),$$(A.10)

where $$\begin{array}{c}{\kappa }_{1,T}={\mathbf{b}}_1^{\mathrm{T}}{\mathbf{O}}_1{({\mathbf{O}}_1^{\mathrm{T}}\mathbf{D}{\mathbf{O}}_1)}^{-1}{\mathbf{O}}_1^{\mathrm{T}}{\mathbf{\Sigma }}_1{\mathbf{O}}_1{({\mathbf{O}}_1^{\mathrm{T}}\mathbf{D}{\mathbf{O}}_1)}^{-1}{\mathbf{O}}_1^{\mathrm{T}}{\mathbf{b}}_1,\\ \mathbf{D}=\left(\begin{array}{cccc}{\mathbf{D}}_1& 0& \cdots & 0\\ 0& {\mathbf{D}}_2& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮\\ 0& 0& \cdots & {\mathbf{D}}_m\end{array}\right),\\ {\mathbf{\Sigma }}_1=\frac{1}{T}\sum _{1\le t,t\prime \le T}\left[\begin{array}{cccc}{\mathbf{D}}_{1,|t-t\prime |}& 0& \cdots & 0\\ 0& {\mathbf{D}}_{2,|t-t\prime |}& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮\\ 0& 0& \cdots & {\mathbf{D}}_{m,|t-t\prime |}\end{array}\right],\\ {\mathbf{D}}_i=E\left[{(\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i})-g{\prime }_{0i}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}){\mathbf{A}}_{0i}{\overline{\mathbf{X}}}_{\mathit{it}})}^{\otimes 2}\right],\\ {\mathbf{D}}_{i,|t-t\prime |}=E[{ϵ}_{\mathit{it}}{ϵ}_{\mathit{it}\prime }(\mathbf{B}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i})-g{\prime }_{0i}({\mathbf{X}}_{\mathit{it}}^{\mathrm{T}}{\mathit{\beta }}_{0i}){\mathbf{A}}_{0i}{\overline{\mathbf{X}}}_{\mathit{it}}){(\mathbf{B}({\mathbf{X}}_{\mathit{it}\prime }^{\mathrm{T}}{\mathit{\beta }}_{0i})-g{\prime }_{0i}({\mathbf{X}}_{\mathit{it}\prime }^{\mathrm{T}}{\mathit{\beta }}_{0i}){\mathbf{A}}_{0i}{\overline{\mathbf{X}}}_{\mathit{it}\prime })}^{\mathrm{T}}].\end{array}$$

A.4  Proof of Theorems 1 and 2

We now consider the proof of Theorems 1 and 2 as special cases of (A.9) and (A.10). Consider first Theorem 2, under the additional assumption that the true partition is used. As shown previously, the asymptotic variance of $\widehat{\overline{\mathit{\beta }}}-{\overline{\mathit{\beta }}}_0$ is $T^{-1}{\mathbf{O}}_2{\mathbf{\Theta }}_2{\mathbf{O}}_2^{\mathrm{T}}$, where ${\mathbf{\Theta }}_2={({\mathbf{O}}_2^{\mathrm{T}}\mathbf{C}{\mathbf{O}}_2)}^{-1}{\mathbf{O}}_2^{\mathrm{T}}{\mathbf{\Sigma }}_2{\mathbf{O}}_2{({\mathbf{O}}_2^{\mathrm{T}}\mathbf{D}{\mathbf{O}}_2)}^{-1}.$ From our proof, it is easy to see that eigenvalues of ${\mathbf{\Theta }}_2$ are bounded and bounded away from zero. By the definition of the $\mathit{mp}\times H_2$ matrix ${\mathbf{O}}_2$, it is easy to see that its row corresponding to β_ij, say denoted by ${\mathbf{O}}_{2(\mathit{ij})}^{\mathrm{T}}$, has a single nonzero entry $1/\sqrt{m_{\mathit{ij}}}$. Let ${\mathbf{e}}_{\mathit{ij}}=\sqrt{m_{\mathit{ij}}}{\mathbf{O}}_{2(\mathit{ij})}$, which is a unit vector, then the asymptotic variance of ${\widehat{\beta }}_{\mathit{ij}}-{\beta }_{0ij}$ is ${(m_{\mathit{ij}}T)}^{-1}{\mathbf{e}}_{\mathit{ij}}^{\mathrm{T}}{\mathbf{\Theta }}_2{\mathbf{e}}_{\mathit{ij}}$.

The asymptotic variance of ${\widehat{\mathit{\theta }}}_i-{\mathit{\theta }}_{0i}$ is $T^{-1}{\mathbf{J}}_i^{G_1}{({\mathbf{D}}^{G_1})}^{-1}{\overline{\mathbf{\Theta }}}_1{({\mathbf{D}}^{G_1})}^{-1}{({\mathbf{J}}_i^{G_1})}^{\mathrm{T}}$, where $${\overline{\mathbf{\Theta }}}_1={({\mathbf{O}}_1^{\mathrm{T}}\mathbf{D}{\mathbf{O}}_1)}^{-1}{\mathbf{O}}_1^{\mathrm{T}}{\mathbf{\Sigma }}_1{\mathbf{O}}_1{({\mathbf{O}}_1^{\mathrm{T}}\mathbf{C}{\mathbf{O}}_1)}^{-1}$$

with eigenvalues bounded and bounded away from zero. By definition of ${\mathbf{J}}_i^{G_1}$ and ${\mathbf{D}}^{G_1}$, it can be seen that each row of the $K\times H_1$ matrix ${\mathbf{J}}_i^{G_1}{({\mathbf{D}}^{G_1})}^{-1}$ has a single nonzero entry $1/\sqrt{m_i}$ and thus if we define ${\mathbf{K}}_i=\sqrt{m_i}{\mathbf{J}}_i^{G_1}{({\mathbf{D}}^{G_1})}^{-1}$, it is easy to directly verify that ${\mathbf{K}}_i^{\mathrm{T}}\mathbf{v}$ is bounded and bounded away from zero and infinity for any unit vector v. Also, we have $\Vert \mathbf{B}(x)\Vert \asymp K$. Thus the asymptotic variance of ${\mathbf{B}}^{\mathrm{T}}(x){\widehat{\mathit{\theta }}}_i-{\mathbf{B}}^{\mathrm{T}}(x){\mathit{\theta }}_{0i}$ can be written as $\frac{K}{m_{i}T}{\mathbf{b}}^{\mathrm{T}}(x){\mathbf{\Theta }}_1^{\mathrm{T}}\mathbf{b}(x)$, if we define $\mathbf{b}(x)={\mathbf{K}}_i^{\mathrm{T}}\mathbf{B}(x)/\Vert {\mathbf{K}}_i^{\mathrm{T}}\mathbf{B}(x)\Vert$, and ${\mathbf{\Theta }}_1={\overline{\mathbf{\Theta }}}_1\Vert {\mathbf{K}}_i^{\mathrm{T}}\mathbf{B}(x){\Vert }^2/K$.

For Theorem 1, since the result is standard, and also is a special case of Theorem 2, we omit the repetition of arguments above. The quantities ${\tilde{\mathbf{e}}}_{\mathit{ij}},\tilde{\mathbf{b}}(x),{\tilde{\mathbf{\Theta }}}_1$ and ${\tilde{\mathbf{\Theta }}}_2$ are defined as above based on the trivial structure in which each single parameter forms its own group in the partition.

The proof of Theorem 2 would be complete if we can establish consistency of homogeneity pursuit based on change point detection. That is, we need to show that the true partition can be identified with probability approaching one.

First, we can show $\Vert {\mathbf{D}}^{G_2}(\widehat{\mathit{\eta }}-{\mathit{\eta }}_0){\Vert }_{\infty }^2=O_p(\hspace{0.17em}\text{log}\hspace{0.17em}(\mathit{Tm})/T)$ and $\Vert {\mathbf{D}}^{G_1}(\widehat{\mathit{\xi }}-{\mathit{\xi }}_0){\Vert }_{\infty }^2=O_p(\hspace{0.17em}\text{log}\hspace{0.17em}(\mathit{Tm})/T)$. The general strategy for establishing these is similar to showing the convergence rate of $\widehat{\mathit{\beta }}$, using a slightly different projection, and one needs to carefully construct bounds that are valid uniformly over components of $\widehat{\mathit{\eta }}$.

Then we use sequence $b_{(1)}\le \cdots \le b_{(n)}(n=\mathit{mp})$ for illustration, with estimated change points ${\widehat{k}}_0=0<{\widehat{k}}_1<\cdots <{\widehat{k}}_{{\widehat{H}}_2}=n$. The true ordered sequence of $\mathit{\beta }$ is ${\beta }_{0(1)}\le \cdots \le {\beta }_{0(n)}$ with change points k_h, $h=0,\dots ,H_2$. Let ${\gamma }_2={\min }_{2\le h\le H_2}|{\beta }_{0(k_{h+1})}-{\beta }_{0(k_h)}|$ be the minimum jump size. The sup-norm convergence results established above, when specializing to the estimator in stage 1, imply that $\Vert \tilde{\mathit{\beta }}-{\mathit{\beta }}_0{\Vert }_{\infty }=O_p(a_T)$ where $a_T=\sqrt{\hspace{0.17em}\text{log}\hspace{0.17em}(\mathit{Tm})/T}$. On the event $\{\Vert \tilde{\mathit{\beta }}-{\mathit{\beta }}_0{\Vert }_{\infty }\le C{a}_T\}$. It is easy to see that (A.11) $$\underset{u-1<k<e}{\max }|{\Delta }_{u,e}(k)-{\Delta }_{u,e}^0(k)|\le a_T.$$(A.11)

where ${\Delta }_{u,e}^0(k)=\sqrt{\frac{(e-k)(k-u+1)}{e-u+1}}\left|\frac{{\sum }_{l=k+1}^e{\beta }_{0(l)}}{e-k}-\frac{{\sum }_{l=u}^k{\beta }_{0(l)}}{k-u+1}\right|.$

Now suppose u – 1 and e are both change points and there is at least one change point inside $(u-1,e)$. Let $\widehat{k}=\arg {\max }_{u-1<k<e}{\Delta }_{u,e}(k)$ and $k_0=\arg {\max }_{u-1<k<e}{\Delta }_{u,e}^0(k)$. We prove consistency by way of contradiction. Suppose $\widehat{k}$ is not one of the true change points. Using some results in Venkatraman (1992) and Cho and Fryzlewicz (2012), we can show that this would lead to ${\Delta }_{u,e}(k_0)>{\Delta }_{u,e}(\widehat{k})$ by (A.11), a contradiction by the definition of $\widehat{k}$. Also, in this case, it is easy to see that ${\max }_{u-1<k<e}{\Delta }_{u,e}(k)\ge {\max }_{u-1<k<e}{\Delta }_{u,e}^0(k)-a_n\ge C{\gamma }_2-a_T>{\delta }_2$.

Now suppose still $u-1,e$ are both change points but there are no other change point inside $(u-1,e)$. In this case, using (A.11), it is easy to see that ${\max }_{u-1<k<e}{\Delta }_{u,e}(k)\le \sqrt{n}{a}_T$.

Since we refrain from further partitioning the interval $(u-1,e)$ if and only if ${\max }_{u-1<k<e}{\Delta }_{u,e}(k)<{\delta }_2$ with $\sqrt{n}{a}_T\ll {\delta }_2\ll {\gamma }_2$, we see that the algorithm consistently identifies exactly the true change points in ${\mathit{\beta }}_0$.

The proof for change point detection in $\mathit{\theta }$ is the same, and the proof of Theorem 2 is complete.

A.5  Proof of Theorem 3

For the first statement, we just need to note that $\overline{\mathit{\beta }}$ is the minimizer of $$\underset{\mathbf{a}}{\min }\sum _{i=1}^m\Vert {\mathit{\beta }}_i-\mathbf{a}{\Vert }^2,$$

and all ${\check{\mathit{\beta }}}_i$ are the same, thus $$\frac{1}{\mathit{mp}}\sum _{i=1}^m\Vert {\check{\mathit{\beta }}}_i-{\mathit{\beta }}_i{\Vert }^2\ge \frac{1}{\mathit{mp}}\sum _{i=1}^m\Vert {\mathit{\beta }}_i-\overline{\mathit{\beta }}{\Vert }^2\ge c.$$

Similarly we can show the second statement.