Efficient GMM estimation with singular system of moment conditions

Abstract

Standard generalised method of moments (GMM) estimation was developed for nonsingular system of moment conditions. However, many important economic models are characterised by singular system of moment conditions. This paper shows that efficient GMM estimation of such models can be achieved by using the reflexive generalised inverses, in particular the Moore–Penrose generalised inverse, of the variance matrix of the sample moment conditions as the weighting matrix. We provide a consistent estimator of the optimal weighting matrix and establish its consistency. Potential issues of using generalised inverse and some remedies are also discussed.

Keywords:

• Singular moment condition models
• generalised method of moments
• reflexive generalised inverses
• imposing random noises

JEL Classifications:

• C13
• C14

1. Introduction

Over the past several decades, a great deal of statisticians' efforts has been devoted to the statistical inference of moment condition models, i.e., models where the linkage between parameter and data is specified through a set of moment restrictions (also known as estimating equations). Technically, a moment condition model specifies that the data generating process of observations $Z_1,\dots ,Z_n$ satisfies (1) $\mathbf{E}\left[g\right(Z_i,{\beta }_0\left)\right]=0,$(1) where g is a ${\mathbb{R}}^K$-valued known function and ${\beta }_0$ is the ${\mathbb{R}}^p$-valued parameter of interest, and $K\ge p$. The popularity of moment condition models is partially due to that a parametric likelihood form may be too strong for many real applications or scientific theories. When the dimension of parameter of interest equals the number of moment conditions, the parameter is said to be just-identified, and the classical approach of the method of moments can be applied for parameter estimation. In practice, a majority of the moment condition models investigated by applied researchers, such as the models for assets pricing and dynamic panel data, are over-identified. The generalised method of moments (GMM) of Hansen (1982) is one of the most popular techniques that are designed for the estimation of over-identified moment condition models (see, e.g., Hansen & West, 2002 and Hall, 2005).

Like many other classical statistical methods, GMM comes with the price of a set of regularity conditions which warrant its validity. Although in most applications those regularity conditions are not binding, some of them can be violated in interesting circumstances. This paper is concerned with the efficient GMM estimation when one of the regularity conditions of standard GMM, that the covariance matrix of the moment vector evaluated at the true parameter be of full rank, is violated. A typical such kind of violation appears when the system of moment conditions is singular, i.e., some components of the moment functions are linear combinations of each other.

Singular systems of moment conditions exist in a wide variety of economic studies, such as the consumer expenditure function analysis (Barten, 1969, 1977), the market share analysis (Rao, 1972; Weiss, 1968), the production function estimation (Dhrymes, 1962), the translog utility function analysis (Berndt & Christensen, 1974), the linearised dynamic stochastic general equilibrium (DSGE) modelling (Bierens, 2007; Ireland, 2004), the errors-in-variables analysis with panel data (Biørn, 2000; Biørn & Klette, 1998; Wansbeek, 2001; Xiao, Shao, & Palta, 2010a, 2010b; Xiao, Shao, Xu, & Palta, 2007), the multivariate random-effects meta-analysis models (Chen, Hong, & Riley,2014; Riley, Abrams, Lambert, Sutton, & Thompson, 2007) and the non-Gaussian ARMA models (Alessi, Barigozzi, & Capasso, 2011; Leeper, Walker, & Yang, 2013; Mountford & Uhlig, 2009; Velasco & Lobato, 2018).

In a linear regression model with known singular disturbance covariance matrix, Theil (1971) showed that a generalised Aitken-like estimator using the Moore–Penrose generalised inverse is best linear unbiased. Following Theil (1971), Kreijger and Neudecker (1977) proposed two optimality criteria to obtain best linear unbiased estimators. Within the same context, Dhrymes and Schwarz (1987) discussed the existence issue of the estimators using generalised inverses. Haupt and Oberhofer (2006) proposed an estimator which does not use the generalised inverses and allows for additional exogenous restrictions, collinearities and generalised adding-up. Bierens and Swanson (2000) and Bierens (2007) suggested that one can obtain parameter estimate by maximising the information content of the singular system. Ireland (2004) and Lai (2008) proposed adding random noises to the singular system to implement maximum likelihood estimation.

In the GMM literature, White (1986) showed that if the estimating function g is of the form $g=(g_1^{\mathrm{\prime }},g_2^{\mathrm{\prime }}{)}^{\mathrm{\prime }}$ such that: (i) ${\mathrm{\Omega }}_1=\mathbf{E}\left[g_1\right(Z_i,{\beta }_0\left)g_1\right(Z_i,{\beta }_0{)}^{\mathrm{\prime }}]$ is nonsingular and (ii) components of $g_2$ are linear combinations of $g_1$, then the efficient GMM estimator is the minimiser of (2) $J_n\left(\beta \right)=n{\overline{g}}_n\left(\beta {)}^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}{\overline{g}}_n\right(\beta ),$(2) where ${\overline{g}}_n\left(\beta \right)=\frac{1}{n}\underset{i=1}{\sum ^n}g(Z_i,\beta )$ and ${\mathrm{\Omega }}^{-}$ is a reflexive generalised inverse of (3) $\mathrm{\Omega }=\mathbf{E}\left[g\right(Z_i,{\beta }_0\left)g\right(Z_i,{\beta }_0{)}^{\mathrm{\prime }}].$(3) However, in practice, the aforementioned representation of g is generally not readily obtainable (see, e.g., Schneeweiss, 2014; Velasco & Lobato, 2018; Xiao et al., 2010b).

The purpose of this article is to develop an efficient GMM estimator for a singular system of moment conditions with general form. An earlier effort appeared in Xiao (2008), which is proposed using the reflexive generalised inverses to deal with the singularity. Schneeweiss (2014) independently discussed similar ideas.

The rest of the paper is organised as follows. In Section 2, we briefly review the GMM methodology, the concepts of generalised inverses and some results of the reflexive generalised inverses. We present our main result in Section 3. Section 4 discusses further issues such as the estimation of optimal weighting matrix and the method of adding noises, and Section 5 concludes. Proofs of results are relegated to the Appendix.

2. GMM and generalised inverses

We first make a brief introduction of the standard GMM method. For book-length detailed account, see Hall (2005). For simplicity we assume that the data $Z_1,\dots ,Z_n$ are i.i.d. Assume also that K>p, i.e., the model is over-identified. Since the number of restrictions on parameter is greater than the dimension of parameter, in general it is impossible to obtain an estimator of the parameter by using method of moments, i.e., by setting the sample moment ${\overline{g}}_n$ equal to zero. The idea of GMM by Hansen (1982) is to minimise a quadratic norm of ${\overline{g}}_n$: (4) $J_n\left(\beta \right)=n{\overline{g}}_n\left(\beta {)}^{\mathrm{\prime }}{W}_n{\overline{g}}_n\right(\beta ),$(4) where $W_n$ is a positive semidefinite matrix. Under a set of regularity conditions including that Ω being positive definite, and assuming $W_n$ converges in probability to a positive semi-definite matrix W, ${\hat{\beta }}_{GMM}$, the minimiser of (4(4) $J_n\left(\beta \right)=n{\overline{g}}_n\left(\beta {)}^{\mathrm{\prime }}{W}_n{\overline{g}}_n\right(\beta ),$(4) ), is a consistent estimator for ${\beta }_0$ and has limiting distribution $\sqrt{n}({\hat{\beta }}_{GMM}-{\beta }_0)\stackrel{d}{\to }N(0,V(W\left)\right),$ where $V\left(W\right)=\left(G^{\mathrm{\prime }}WG{)}^{-1}{G}^{\mathrm{\prime }}W\mathrm{\Omega }WG\right(G^{\mathrm{\prime }}WG{)}^{-1}$ with $G=E\left[\frac{\mathrm{\partial }g(Z_i,{\beta }_0)}{\mathrm{\partial }\beta }\right]$. The lower bound of $V\left(W\right)$ is achieved at $W={\mathrm{\Omega }}^{-1}$, i.e., $V\left(W\right)\ge V\left({\mathrm{\Omega }}^{-1}\right)$ in the sense of being nonnegative definite, for any W. In practice, a consistent estimator of ${\mathrm{\Omega }}^{-1}$ can be set as (5) ${\hat{\mathrm{\Omega }}}^{-1}={\left[\frac{1}{n}\underset{i=1}{\sum ^n}\left[g\right(Z_i,\tilde{\beta }\left)g\right(Z_i,\tilde{\beta }{)}^{\mathrm{\prime }}]\right]}^{-1},$(5) where $\tilde{\beta }$ is a consistent estimator of ${\beta }_0$. A typical choice of $\tilde{\beta }$ is a GMM estimator with $W=I_K$, the identity matrix of order K. Note that ${\hat{\mathrm{\Omega }}}^{-1}$ converges in probability to ${\mathrm{\Omega }}^{-1}$ because $\frac{1}{n}\underset{i=1}{\sum ^n}\left[g\right(Z_i,\tilde{\beta }\left)g\right(Z_i,\tilde{\beta }{)}^{\mathrm{\prime }}]$ converges in probability to Ω, and more importantly, Ω is positive definite.

Next we review the concepts of generalised inverses of a matrix and some of their properties.

Definition 2.1

Let A be a real $l\times s$ matrix. An $s\times l$ real matrix $A^{-}$ may have one or all of the following properties:

1. $A{A}^{-}A=A$;

2. $A^{-}A{A}^{-}=A^{-}$;

3. $(A{A}^{-}{)}^{\mathrm{\prime }}=A{A}^{-}$;

4. $(A^{-}A{)}^{\mathrm{\prime }}=A^{-}A$.

If $A^{-}$ satisfies (i), it is called a generalised inverse of A; if $A^{-}$ satisfies (i) and (ii), it is called a reflexive generalised inverse (or ${\mathbf{g}}_2$-inverse) of A; if $A^{-}$ satisfies (i) –(iv), it is called the Moore–Penrose generalised inverse of A. The Moore–Penrose generalised inverse of a matrix A is unique and is denoted by $A^{+}$ hereafter.1

We list some of the important properties of the generalised inverses by the following two propositions, proof of which can be achieved by direct verification and therefore is omitted.2 Proposition 1 states that when the matrix of interest has natural factorisation with certain structure, some of its generalised inverses can be easily derived.

Proposition 2.1

(i) Let $\mathrm{\Omega }=\left[A_1^{\prime }{A}_2^{\prime }{]}^{\prime }{\mathrm{\Omega }}_1\right[A_1^{\prime }{A}_2^{\prime }]$, where $A_1$ and ${\mathrm{\Omega }}_1$ are nonsingular square matrices. Then ${\mathrm{\Omega }}^{-}=\left[\begin{array}{cc}(A_1^{-1}{)}^{\mathrm{\prime }}{\mathrm{\Omega }}_1^{-1}{A}_1^{-1}& 0\\ 0& 0\end{array}\right]$ is a reflexive generalised inverse of Ω.

(ii) Let $\mathrm{\Omega }=A{\mathrm{\Omega }}_1{A}^{\mathrm{\prime }}$, where A is of full column rank and ${\mathrm{\Omega }}_1$ is nonsingular, then any reflexive generalised inverse ${\mathrm{\Omega }}^{-}$ of Ω satisfies $A^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}A={\mathrm{\Omega }}_1^{-1}$. Moreover, we have $A^{+}=(A^{\mathrm{\prime }}A{)}^{-1}{A}^{\mathrm{\prime }}$, and ${\mathrm{\Omega }}^{+}=(A^{+}{)}^{\mathrm{\prime }}{\mathrm{\Omega }}_1^{-1}{A}^{+}=A(A^{\mathrm{\prime }}A{)}^{-1}{\mathrm{\Omega }}_1^{-1}(A^{\mathrm{\prime }}A{)}^{-1}{A}^{\mathrm{\prime }}.$

Proposition 2.2 points out that the generalised inverses (including the reflexive generalised inverses) are not unique and can be obtained by using the singular value decomposition .3

Proposition 2.2

Let Ω be an $m\times n$ real valued matrix with rank r>0. Suppose that the singular value decomposition of Ω is $\mathrm{\Omega }=S\mathrm{\Sigma }{T}^{\prime }$, where S is $m\times m$ with $S^{\prime }S=I_m$, T is $n\times n$ with $T^{\prime }T=I_n$, and $\mathrm{\Sigma }=\left[\begin{array}{cc}{\mathrm{\Sigma }}_r& O\\ O& O\end{array}\right]$, with ${\mathrm{\Sigma }}_r$ the diagonal matrix of singular values of Ω and O the matrices of zeros. Then

(i) (6) $G=T\left[\begin{array}{cc}{\mathrm{\Sigma }}_r^{-1}& X\\ Y& Z\end{array}\right]S^{\prime }$(6) is a generalised inverse of Ω, where X, Y and Z are arbitrary real valued matrices with appropriate dimension.

(ii) (7) $G=T\left[\begin{array}{cc}{\mathrm{\Sigma }}_r^{-1}& X\\ Y& Y{\mathrm{\Sigma }}_r^{-1}X\end{array}\right]S^{\prime }$(7) is a reflexive generalised inverse of Ω, where X, Y are arbitrary real valued matrices with appropriate dimension.

(iii) (8) $G=T\left[\begin{array}{cc}{\mathrm{\Sigma }}_r^{-1}& O\\ O& O\end{array}\right]S^{\prime }$(8) is the Moore–Penrose generalised inverse of Ω.

White (1986) result on GMM estimation with singular moment conditions can be stated as follows:

Theorem 2.1

White, 1986

Suppose there exists a matrix Δ such that $G=[I_{l_1}\mathrm{\Delta }{]}^{\prime }{G}_1$ and $\mathrm{\Omega }=\left[I_{l_1}\mathrm{\Delta }{]}^{\prime }{\mathrm{\Omega }}_1\right[I_{l_1}\mathrm{\Delta }]$, where $G_1$ is of full column rank and ${\mathrm{\Omega }}_1$ is $l_1\times l_1$ positive definite with $l_1\ge p$, then

(i) For any reflexive generalised inverse ${\mathrm{\Omega }}^{-}$ of Ω, $\mathrm{\Omega }-G(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}{G}^{\mathrm{\prime }}$ is independent of the choice of ${\mathrm{\Omega }}^{-}$, and $\mathrm{\Omega }-G(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}{G}^{\mathrm{\prime }}\ge 0$.

(ii) For any reflexive generalised inverse ${\mathrm{\Omega }}^{-}$ of Ω, and for any W, $\left(G^{\mathrm{\prime }}WG{)}^{-1}{G}^{\mathrm{\prime }}W\mathrm{\Omega }WG\right(G^{\mathrm{\prime }}WG{)}^{-1}\ge (G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}.$ Hence ${\mathrm{\Omega }}^{-}$ is the optimal weighting matrix. In practice, one may choose the Moore–Penrose generalised inverse of Ω, or a special reflexive generalised inverse ${\mathrm{\Omega }}^{-}=\left[\begin{array}{cc}{\mathrm{\Omega }}_1^{-1}& 0\\ 0& 0\end{array}\right].$

Remark

Note that $(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}$ – the asymptotic covariance matrix of the optimal GMM estimator – does not depend on Δ. The basic idea of Theorem 2.1 is as follows. Suppose we have two sets of instrumental variables, say $Z_1$ and $Z_2$, such that $Z_1$ is linearly independent and $Z_2$ is a linear combination of $Z_1$, i.e., $Z_2=Z_1\alpha$ for some constant vector α, then one can ignore $Z_2$ and use $Z_1$ only as instruments, in doing so we achieve the same asymptotic efficiency as using $Z=\left(Z_1{Z}_2\right)$. The limitation of this result is that to apply this method we have to sort all instrumental variables into two groups, such that instrumental variables in one group are linear combinations of those in the other group. This can be very tedious in practice. For example, in panel data models, there are often a very large number of instruments and it is in general impossible to sort out them.

3. Main results

We now establish some basic results about random vectors with singular covariance matrices.

Lemma 3.1

Let $Y=[Y_1\dots Y_m{]}^{\mathrm{\prime }}$ be an $m\times 1$ random vector and r be the rank of the covariance matrix of Y. Suppose that r<m. Then

(i) There exist a r-dimensional subvector $Y^{\left(r\right)}=[Y_{i_1}\dots Y_{i_r}{]}^{\mathrm{\prime }}$ of Y such that its covariance matrix $\mathbf{v}\mathbf{a}\mathbf{r}\left(Y^{\left(r\right)}\right)$ is positive definite. The vector $Y^{\left(r\right)}$ is called an essential subvector of Y.

(ii) Let $Y^{-\left(r\right)}$ be the $(m-r)\times 1$ vector consisting of the remaining components of Y. Then there exist an $(m-r)\times (m-r)$ constant matrix C and an $(m-r)\times 1$ constant vector d such that $Y^{-\left(r\right)}=C{Y}^{\left(r\right)}+d,\mathrm{w}.\mathrm{p}.1.,$ where w.p.1. means ‘with probability one’. Hence there exist an $m\times r$ constant matrix B of full column rank and an $m\times 1$ constant vector $\tilde{d}$ such that $Y=B{Y}^{\left(r\right)}+\tilde{d},\mathrm{w}.\mathrm{p}.1.$ (iii) If $\mathbf{E}Y=0$, then $\tilde{d}$ in (ii) is the zero vector. i.e., $Y=B{Y}^{\left(r\right)}$, w.p.1.

Theorem 3.1 is the main result of this paper.

Theorem 3.1

Consider GMM estimation for model (1(1) $\mathbf{E}\left[g\right(Z_i,{\beta }_0\left)\right]=0,$(1) ) with Ω defined by (3(3) $\mathrm{\Omega }=\mathbf{E}\left[g\right(Z_i,{\beta }_0\left)g\right(Z_i,{\beta }_0{)}^{\mathrm{\prime }}].$(3) ). Suppose it is known that the components of $g(Z_i,{\beta }_0)$ are linearly dependent (with probability one). Then any reflexive inverse of Ω is an optimal weighting matrix. Particularly, we can use the Moore–Penrose generalised inverse ${\mathrm{\Omega }}^{+}$.

Let ${\mathrm{\Omega }}^{-}$ be an arbitrary reflexive generalised inverse of Ω. Then the asymptotic variance matrix of the GMM estimator using ${\mathrm{\Omega }}^{-}$ as the weighting matrix is $\begin{array}{rl}V\left({\mathrm{\Omega }}^{-}\right)& =\left(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}{G}^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}\mathrm{\Omega }{\mathrm{\Omega }}^{-}G\right(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}\\ & =(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1},\end{array}$ where $G=\mathbf{E}\left[\frac{\mathrm{\partial }g(Z_i,{\beta }_0)}{\mathrm{\partial }\beta }\right]$. A natural question is whether $V\left({\mathrm{\Omega }}^{-}\right)$ is a constant matrix independent of the choice of ${\mathrm{\Omega }}^{-}$. The answer is yes. To see this, suppose the essential subvector of $g(Z_i,{\beta }_0)$ is $g^{\left(r\right)}(Z_i,{\beta }_0)=\left(g_{i_1}\right(Z_i,{\beta }_0),\dots ,g_{i_r}(Z_i,{\beta }_0){)}^{\mathrm{\prime }}$, with $g(Z_i,{\beta }_0)=B{g}^{\left(r\right)}(Z_i,{\beta }_0)(w.p.1.)$. Let $G_1=\mathbf{E}\left[\frac{\mathrm{\partial }{g}^{\left(r\right)}(Z_i,{\beta }_0)}{\mathrm{\partial }{\beta }^{\mathrm{\prime }}}\right]$ and ${\mathrm{\Omega }}_1=\mathbf{v}\mathbf{a}\mathbf{r}\left(g^{\left(r\right)}\right(Z_i,{\beta }_0\left)\right)$. Then we have $G=B{G}_1$, $B^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}B={\mathrm{\Omega }}_1^{-1}$, hence $\begin{array}{rl}V\left({\mathrm{\Omega }}^{-}\right)& =(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}=(G_1^{\mathrm{\prime }}{B}^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}B{G}_1{)}^{-1}\\ & =(G_1^{\mathrm{\prime }}{\mathrm{\Omega }}_1^{-1}{G}_1{)}^{-1},\end{array}$ which is independent of the choice of ${\mathrm{\Omega }}^{-}$, as the essential vector and the corresponding matrices $G_1$ and ${\mathrm{\Omega }}_1$ are unrelated to ${\mathrm{\Omega }}^{-}$. We also see that $V\left({\mathrm{\Omega }}^{-}\right)$ remains the same if we use another essential vector, as $(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}=(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{+}G{)}^{-1}$ is unrelated to the choice of the essential vectors. More details can be found in the proof of Theorem 3.1 in the Appendix.

Judging from the asymptotic distributions we can see that GMM estimation using moment conditions (1(1) $\mathbf{E}\left[g\right(Z_i,{\beta }_0\left)\right]=0,$(1) ) and the reflexive generalised inverses as weighting matrix is asymptotically equivalent to the efficient GMM using moment conditions $\mathbf{E}\left[g^{\left(r\right)}\right(Z_i,{\beta }_0\left)\right]=0$. In some situations, one can figure out the essential subvector $g^{\left(r\right)}(Z_i,\beta )$, then efficient GMM estimation can be based on $\mathbf{E}\left[g^{\left(r\right)}\right(Z_i,{\beta }_0\left)\right]=0$ directly. For instance, in the errors-in-variables analysis of panel data, Xiao et al. (2010a) and Xiao et al. (2010b) found that one can obtain $g^{\left(r\right)}(Z_i,\beta )$ by using singular value decomposition. However, such simple decomposition is not available in general and it can be very inconvenient, if not impossible, to find the essential vector $g^{\left(r\right)}(Z_i,\beta )$. Theorem 2 tells us that whatever this subvector is, the GMM estimator using any of the reflexive generalised inverses, and the Moore–Penrose generalised inverse in particular, as the weighting matrix will always have the same asymptotic variance as the efficient GMM based on $\mathbf{E}\left[g^{\left(r\right)}\right(Z_i,{\beta }_0\left)\right]=0$.

4. Further issues

4.1. Optimal weighting matrix estimation

Now we discuss consistent estimation of ${\mathrm{\Omega }}^{+}$. Let $\tilde{\beta }$ be a consistent estimator of ${\beta }_0$ and ${\hat{\mathrm{\Omega }}}_n=\frac{1}{n}\underset{i=1}{\sum ^n}\left[g\right(Z_i,\tilde{\beta }\left)g\right(Z_i,\tilde{\beta }{)}^{\mathrm{\prime }}]$. Then ${\hat{\mathrm{\Omega }}}_n\to \mathrm{\Omega }$ in probability under normal regularity conditions. A natural candidate estimator of ${\mathrm{\Omega }}^{+}$ is ${\hat{\mathrm{\Omega }}}_n^{+}$.

It is well known that if a sequence of nonsingular square matrices $\left\{A_n\right\}$ converges to a nonsingular square matrix A, then $A_n^{-1}\to A^{-1}$.4 However, if A is singular, and $A_n\to A$, we may not necessarily have that $A_n^{+}\to A^{+}$. 5 Assuming $A_n\to A$ and A is singular, then a necessary and sufficient condition for $A_n^{+}\to A^{+}$ is:

Theorem 4.1

Stewart, 1969

Let $\left\{A_n\right\}$ be a sequence of real $m\times n$ matrices converging to a $m\times n$ matrix A. Then $A_n^{+}\to A^{+}$ if and only $rank\left(A_n\right)=rank\left(A\right)$ for n large enough.

We now prove that ${\hat{\mathrm{\Omega }}}_n^{+}$ is a consistent estimator for ${\mathrm{\Omega }}^{+}$. By Theorem 4.1, we need only to show that $rank\left({\hat{\mathrm{\Omega }}}_n\right)=rank\left(\mathrm{\Omega }\right)$ when n is large enough. By Lemma 3.1, there exists a constant matrix B of full column rank, such that w.p.1., $\begin{array}{rl}{\hat{\mathrm{\Omega }}}_n& =\frac{1}{n}\underset{i=1}{\sum ^n}\left[B{g}^r\right(Z_i,\tilde{\beta }\left)g^{\left(r\right)}\right(Z_i,\tilde{\beta }{)}^{\mathrm{\prime }}{B}^{\mathrm{\prime }}]\\ & =B\left(\frac{1}{n}\underset{i=1}{\sum ^n}\left[g^{\left(r\right)}\right(Z_i,\tilde{\beta }\left)g^{\left(r\right)}\right(Z_i,\tilde{\beta }{)}^{\mathrm{\prime }}]\right)B^{\mathrm{\prime }}.\end{array}$ Since the components of $g^{\left(r\right)}(Z_i,\beta )$ are linear independent for any β, $rank\left(\frac{1}{n}\underset{i=1}{\sum ^n}\left[g^{\left(r\right)}\right(Z_i,\hat{\beta }\left)g^{\left(r\right)}\right(Z_i,\hat{\beta }{)}^{\mathrm{\prime }}]\right)=r=rank\left(\mathrm{\Omega }\right),$ for any n. Therefore $rank\left({\hat{\mathrm{\Omega }}}_n\right)=rank\left(\frac{1}{n}\underset{i=1}{\sum ^n}\right[g^{\left(r\right)}(Z_i,\hat{\beta })g^{\left(r\right)}(Z_i,\hat{\beta }{)}^{\mathrm{\prime }}])=rank\left(\mathrm{\Omega }\right)$, and ${\hat{\mathrm{\Omega }}}_n^{+}$ converges to ${\mathrm{\Omega }}^{+}$ in probability.

Even though using generalised inverses is theoretically sound, it can be unstable, i.e., small perturbation of a singular matrix may result in large deviation from its generalised inverses.6 Therefore, one must be cautious when using generalised inverses. We suggest that one should first try to find the essential subvector $g^{\left(r\right)}$. In case that $g^{\left(r\right)}$ is not easily obtainable, the method introduced below can be used as an alternative to generalised inverses.

4.2. Imposing random noises

To avoid the potential bias caused by generalised inverses, we can add randomly generated noises to the system to make it nonsingular, as Bierens (2007) and Lai (2008) did in the maximum likelihood estimation of singular system of equations. Specifically, let $U_1,\dots ,U_n$ be i.i.d. $K\times 1$ random vectors generated from the multivariate normal distribution with mean zero and covariance matrix ${\sigma }^2{I}_K$, and assume that $U_1,\dots ,U_n$ are independent from $Z_1,\dots ,Z_n$. Define $h(Z_i,Ui,\beta )=g(Z_i,\beta )+U_i$, for $i=1,\dots ,n$. Then ${\beta }_0$ is the solution of the set of moment conditions (9) $\mathbf{E}\left[h\right(Z_i,U_i,\beta \left)\right]=0.$(9) The set of moment conditions (9(9) $\mathbf{E}\left[h\right(Z_i,U_i,\beta \left)\right]=0.$(9) ) is nonsingular, since $\mathrm{\Sigma }=\mathbf{E}\left[h\right(Z_i,U_i,{\beta }_0\left)h\right(Z_i,U_i,{\beta }_0{)}^{\prime }]=\mathrm{\Omega }+{\sigma }^2{I}_K>0$. Let ${\tilde{\beta }}_{GMM}$ be an efficient GMM estimator of ${\beta }_0$ based on (9(9) $\mathbf{E}\left[h\right(Z_i,U_i,\beta \left)\right]=0.$(9) ), then the asymptotic distribution of ${\tilde{\beta }}_{GMM}$ is $\sqrt{n}({\tilde{\beta }}_{GMM}-{\beta }_0)\stackrel{d}{\to }N(0,(G^{\prime }{\mathrm{\Sigma }}^{-1}G{)}^{-1})$. Since $(G^{\prime }{\mathrm{\Sigma }}^{-1}G{)}^{-1}>(G^{\prime }{\mathrm{\Omega }}^{-1}G{)}^{-1}$, for any $\sigma >0$, ${\tilde{\beta }}_{GMM}$ is asymptotically less efficient than ${\hat{\beta }}_{GMM}$. However, the loss of efficiency can be controlled since $(G^{\prime }{\mathrm{\Sigma }}^{-1}G{)}^{-1}\to (G^{\prime }{\mathrm{\Omega }}^{-1}G{)}^{-1}$, as $\sigma \to 0$. Similar to Lai (2008), one can also generate m independent samples of $U_1,\dots ,U_n$, obtain m GMM estimators ${\tilde{\beta }}_{GMM}^1,\dots ,{\tilde{\beta }}_{GMM}^m$ and then construct a new estimator by ${\tilde{\beta }}_{GMM}^A=\frac{1}{m}\underset{j=1}{\sum ^m}{\tilde{\beta }}_{GMM}^j$.7 Since ${\tilde{\beta }}_{GMM}^A$ combines information in ${\tilde{\beta }}_{GMM}^1,\dots ,{\tilde{\beta }}_{GMM}^m$, in theory it is asymptotically more efficient than any of ${\tilde{\beta }}_{GMM}^1,\dots ,{\tilde{\beta }}_{GMM}^m$. It is of interest to investigate the asymptotic distribution and finite sample performance of ${\tilde{\beta }}_{GMM}^A$ in a future study.

5. Concluding remarks

Since the moment condition models do not require researchers to specify the likelihood function of the data generating process, they have been widely used by econometricians to model economic theories. Though it is desirable that the moment conditions constructed from economic theory are linearly independent, in practice this may not always be the case. Sometimes singularity is inherent in the model or is caused by some singular transformations. In this paper, we extended the efficient GMM estimation to linearly dependent moment condition models. The result can be viewed as a natural extension of the standard GMM theory, since the generalised inverse of a matrix is a natural extension of the inverse of a matrix. Though in theory using generalised inverses yields efficient GMM estimators, in practice one must be cautious of using them, in light of the following two concerns. First, using generalised inverses ignores the intrinsic structure of the moment conditions, which sometimes contains important information. Second, the generalised inverses of a singular matrix are unstable, which could induce serious bias of the resulting GMM estimator. Therefore when there is singularity in the system, a practical strategy is to obtain an essential moment vector and apply GMM to it. In case an essential moment vector is not available, we can add random noises to the moment conditions and obtain GMM estimators based on the new set of moment conditions. We suggest using generalised inverses with discretion. The results in this paper might also shed light on other popular statistical methods (such as the empirical likelihood) for estimating equations with singularity.

Disclosure statement

No potential conflict of interest was reported by the author.

Additional information

Funding

The research is supported by the National Natural Science Foundation of China (NSFC grant: 71661137005, 71473040 and 11571081).

Notes on contributors

Zhiguo Xiao

Dr. Zhiguo Xiao received his PhD in statistics from the University of Wisconsin-Madison. His research interest includes both theoretical and applied econometrics.

Notes

3 Similar results for square matrices can be found in Bapat (2012, pp. 47–48).

2 More results on reflexive generalised inverses can be found in Rao and Mitra (1971), Rao (2001), Bapat (2012), Fampa and Lee (2018), and Xu, Fampa, and Lee (2019).

1 For the existence and uniqueness of the Moore–Penrose generalised inverse, see, e.g., Penrose (1955) and Abadir and Magnus (2005, pp. 284–285).

5 For example, consider $A_n=\left[\begin{array}{cc}1-\frac{1}{n}& 1-\frac{1}{n^2}\\ 1-\frac{1}{n^2}& 1\end{array}\right]$ and $A=\left[\begin{array}{cc}1& 1\\ 1& 1\end{array}\right]$. Then $A_n\to A$. Since $A_n$ is invertible, $A_n^{+}=A_n^{-1}\to \left[\begin{array}{cc}-\mathrm{\infty }& \mathrm{\infty }\\ \mathrm{\infty }& -\mathrm{\infty }\end{array}\right]$. Hence $A_n^{+}\nrightarrow A^{+}=\left[\begin{array}{cc}\frac{1}{2}& \frac{1}{2}\\ \frac{1}{2}& \frac{1}{2}\end{array}\right]$.

4 A sequence of real $m\times n$ matrices $\left\{A_n\right\}$ is said to converge to a $m\times n$ matrix A if $\parallel A_n-A\parallel \to 0$, where $\parallel \cdot \parallel$ is a matrix norm, such as the Euclidean norm or $\parallel A\parallel =\underset{\parallel x\parallel =1}{sup}\{\parallel Ax\parallel \}$.

6 For example, consider a matrix $A=\left[\begin{array}{ccc}1& & \\ & 0& \\ & & 0\end{array}\right]$. Its Moore–Penrose generalised inverse is $A^{+}=A$. Adding a small number $\epsilon ={10}^{-3}$ to the first two diagonal entries of A, we obtain a new Moore–Penrose generalised inverse $\left[\begin{array}{ccc}0.999& & \\ & 1000& \\ & & 0\end{array}\right]$.

7 This idea is similar to bootstrap aggregating (bagging).

Appendix

Proofs of results

Proof

Proof of Lemma 1

Let $\mathrm{\Gamma }=\mathbf{v}\mathbf{a}\mathbf{r}\left(Y\right)$, and $\mathrm{\Gamma }=T\left(\begin{array}{cc}{\mathrm{\Lambda }}_r& 0\\ 0& 0\end{array}\right)T^{\mathrm{\prime }}$ be the spectrum decomposition of Γ, with $T^{\mathrm{\prime }}T=T{T}^{\mathrm{\prime }}=I_m$. Let $T=\left[T_1{T}_2\right]$, then $\begin{array}{rl}{T}^{\mathrm{\prime }}T& =\left[\begin{array}{c}{T}_1^{\mathrm{\prime }}\\ T_2^{\mathrm{\prime }}\end{array}\right]\left[T_1{T}_2\right]=\left[\begin{array}{cc}{T}_1^{\mathrm{\prime }}{T}_1& T_1^{\mathrm{\prime }}{T}_2\\ T_2^{\mathrm{\prime }}{T}_1& T_2^{\mathrm{\prime }}{T}_2\end{array}\right]=I_m,\\ \mathrm{\Gamma }& =T\left[\begin{array}{cc}{\mathrm{\Lambda }}_r& 0\\ 0& 0\end{array}\right]T^{\mathrm{\prime }}=T_1{\mathrm{\Lambda }}_r{T}_1^{\mathrm{\prime }},\end{array}$ hence $T_2^{\mathrm{\prime }}{T}_1=0$, and $\mathbf{v}\mathbf{a}\mathbf{r}\left(T_2^{\mathrm{\prime }}Y\right)=T_2^{\mathrm{\prime }}V{T}_2=T_2^{\mathrm{\prime }}{T}_1{\mathrm{\Lambda }}_r{T}_1^{\mathrm{\prime }}{T}_2=0$. Hence there exists an $(m-r)\times 1$ constant vector c such $T_2^{\mathrm{\prime }}Y=c$. Let $T_2^{\mathrm{\prime }}=[{\mathbf{t}}_1\dots {\mathbf{t}}_m]$, with $rank\left(T_2^{\mathrm{\prime }}\right)=q:=m-r$. Suppose ${\mathbf{t}}_{j_1}\dots$ ${\mathbf{t}}_{j_q}$ are linearly independent, then $\begin{array}{rl}{T}_2^{\mathrm{\prime }}Y& =[{\mathbf{t}}_1\cdots {\mathbf{t}}_m]{[Y_1\cdot Y_m]}^{\mathrm{\prime }}\\ & ={\mathbf{t}}_1{Y}_1+\cdots +{\mathbf{t}}_m{Y}_m\\ & =[{\mathbf{t}}_{j_1}\dots {\mathbf{t}}_{j_q}]{[Y_{j_1}\cdot Y_{j_q}]}^{\mathrm{\prime }}+\hat{T}{Y}^{-\left(q\right)},\end{array}$ hence $[{\mathbf{t}}_{j_1}\dots {\mathbf{t}}_{j_q}][Y_{j_1}\cdot Y_{j_q}{]}^{\mathrm{\prime }}+\hat{T}{Y}^{-\left(q\right)}=c$, i.e., $Y^{\left(q\right)}={[Y_{j_1}\cdot Y_{j_q}]}^{\mathrm{\prime }}=-{[{\mathbf{t}}_{j_1}\dots {\mathbf{t}}_{j_q}]}^{-1}\hat{T}{Y}^{-\left(q\right)}+{[{\mathbf{t}}_{j_1}\dots {\mathbf{t}}_{j_q}]}^{-1}c.$ Let $C=-[{\mathbf{t}}_{j_1}\dots {\mathbf{t}}_{j_q}{]}^{-1}\hat{T},d=[{\mathbf{t}}_{j_1}\dots {\mathbf{t}}_{j_q}{]}^{-1}c$, we get $Y^{\left(q\right)}=C{Y}^{-\left(q\right)}+d$, i.e., $Y^{-\left(r\right)}=C{Y}^{\left(r\right)}+d$. Since $Y^{\left(r\right)}$ and $Y^{-\left(r\right)}$ are subvectors of Y, there exists a $m\times m$ nonsingular matrix A such that $Y=A\left[\begin{array}{c}{Y}^{\left(r\right)}\\ Y^{-\left(r\right)}\end{array}\right]=A\left[\begin{array}{c}{Y}^{\left(r\right)}\\ C{Y}^{\left(r\right)}+d\end{array}\right]=A\left[\begin{array}{c}I\\ C\end{array}\right]Y^r+A\left[\begin{array}{c}0\\ d\end{array}\right],$ i.e., $Y=B{Y}^{\left(r\right)}+\tilde{d}$, with $B=A\left[\begin{array}{c}I\\ C\end{array}\right],\tilde{d}=A\left[\begin{array}{c}0\\ d\end{array}\right]$. Hence $\mathbf{v}\mathbf{a}\mathbf{r}\left(Y\right)=B^{\mathrm{\prime }}\mathbf{v}\mathbf{a}\mathbf{r}\left(Y^{\left(r\right)}\right)B$. Since $B=A\left[\begin{array}{c}I\\ C\end{array}\right]$ is of full column rank, $rank\left(\mathbf{v}\mathbf{a}\mathbf{r}\right(Y\left)\right)=rank\left(\mathbf{v}\mathbf{a}\mathbf{r}\right(Y^{\left(r\right)}\left)\right)=r$. This shows that $\mathbf{v}\mathbf{a}\mathbf{r}\left(Y^{\left(r\right)}\right)$ is nonsingular.

Proof

Proof of Theorem 3.1

Let $V\left(W\right)$ denote the asymptotic variance of the GMM estimator using weighting matrix W. Then $V\left(W\right)=\left(G^{\mathrm{\prime }}WG{)}^{-1}{G}^{\mathrm{\prime }}W\mathrm{\Omega }WG\right(G^{\mathrm{\prime }}WG{)}^{-1}$. Let ${\mathrm{\Omega }}^{-}$ be a reflexive generalised inverse of Ω. Then we have $\begin{array}{rl}V\left({\mathrm{\Omega }}^{-}\right)& =\left(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}{G}^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}\mathrm{\Omega }{\mathrm{\Omega }}^{-}G\right(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}\\ & =(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}.\end{array}$ Hence $V\left(W\right)-V\left({\mathrm{\Omega }}^{-}\right)=\left(G^{\mathrm{\prime }}WG{)}^{-1}{G}^{\mathrm{\prime }}W\right[\mathrm{\Omega }-G\left(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}{G}^{\mathrm{\prime }}\right]WG(G^{\mathrm{\prime }}WG{)}^{-1}$. To establish $V\left(W\right)-V\left({\mathrm{\Omega }}^{-}\right)\ge 0$, we just need to show that $\mathrm{\Omega }-G(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}{G}^{\mathrm{\prime }}\ge 0$. Let $rank\left(\mathrm{\Omega }\right)=r$. By Lemma 3.1, there exist a subvector $g^{\left(r\right)}(Z_i,{\beta }_0)=\left(g_{i_1}\right(Z_i,{\beta }_0),\dots ,g_{i_r}(Z_i,{\beta }_0){)}^{\mathrm{\prime }}$ and a matrix B of full column rank such that $g(Z_i,{\beta }_0)=B{g}^r(Z_i,{\beta }_0)$ a.s., with ${\mathrm{\Omega }}_1=\mathbf{v}\mathbf{a}\mathbf{r}\left(g^{\left(r\right)}\right(Z_i,{\beta }_0\left)\right)$ positive definite. Then $\mathrm{\Omega }=\mathbf{v}\mathbf{a}\mathbf{r}\left(g\right(Z_i,{\beta }_0\left)\right)=B{\mathrm{\Omega }}_1{B}^{\mathrm{\prime }}$, and $G=\mathbf{E}\left[\frac{\mathrm{\partial }g(Z_i,{\beta }_0)}{\mathrm{\partial }\beta }\right]=B{G}_1$, with $G_1=\mathbf{E}\left[\frac{\mathrm{\partial }{g}^{\left(r\right)}(Z_i,{\beta }_0)}{\mathrm{\partial }{\beta }^{\mathrm{\prime }}}\right]$. Hence $\begin{array}{rl}\mathrm{\Omega }-G(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}{G}^{\mathrm{\prime }}& =B{\mathrm{\Omega }}_1{B}^{\mathrm{\prime }}-B{G}_1(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}{G}_1^{\mathrm{\prime }}{B}^{\mathrm{\prime }}\\ & =B[{\mathrm{\Omega }}_1-G_1(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}{G}_1^{\mathrm{\prime }}]B^{\mathrm{\prime }}.\end{array}$So we just need to show that ${\mathrm{\Omega }}_1-G_1(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}{G}_1^{\mathrm{\prime }}\ge 0$. By Proposition 2.1, $B^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}B={\mathrm{\Omega }}_1^{-1}$, hence $\begin{array}{rl}& {\mathrm{\Omega }}_1-G_1(G^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}G{)}^{-1}{G}_1^{\mathrm{\prime }}\\ & ={\mathrm{\Omega }}_1-G_1(G_1^{\mathrm{\prime }}{B}^{\mathrm{\prime }}{\mathrm{\Omega }}^{-}B{G}_1{)}^{-1}{G}_1^{\mathrm{\prime }}\\ & ={\mathrm{\Omega }}_1-G_1(G_1^{\mathrm{\prime }}{\mathrm{\Omega }}_1^{-1}{G}_1{)}^{-1}{G}_1^{\mathrm{\prime }}\\ & ={\mathrm{\Omega }}_1^{\frac{1}{2}}[I-{\mathrm{\Omega }}_1^{-\frac{1}{2}}{G}_1(G_1^{\mathrm{\prime }}{\mathrm{\Omega }}_1^{-1}{G}_1{)}^{-1}{G}_1^{\mathrm{\prime }}{\mathrm{\Omega }}_1^{-\frac{1}{2}}]{\mathrm{\Omega }}_1^{\frac{1}{2}}\\ & \ge 0,\end{array}$ since $I-{\mathrm{\Omega }}_1^{-\frac{1}{2}}{G}_1(G_1^{\mathrm{\prime }}{\mathrm{\Omega }}_1^{-1}{G}_1{)}^{-1}{G}_1^{\mathrm{\prime }}{\mathrm{\Omega }}_1^{-\frac{1}{2}}$ is idempotent and symmetric.