A restricted gamma ridge regression estimator combining the gamma ridge regression and the restricted maximum likelihood methods of estimation

Abstract

In this article, we propose a restricted gamma ridge regression estimator (RGRRE) by combining the gamma ridge regression (GRR) and restricted maximum likelihood estimator (RMLE) to combat multicollinearity problem for estimating the parameter $\beta$ in the gamma regression model. The properties of the new estimator are discussed, and its superiority over the GRR, RMLE and traditional maximum likelihood estimator is theoretically analysed under different conditions. We also suggest some estimating methods to find the optimal value of the shrinkage parameter. A Monte Carlo simulation study is conducted to judge the performance of the proposed estimator. Finally, an empirical application is analysed to show the benefit of RGRRE over the existing estimators.

KEYWORDS:

• Gamma regression model
• maximum likelihood estimator
• multicollinearity
• mean squared error
• restricted gamma ridge regression estimator

1. Introduction

Gamma regression model (GRM) is frequently used in the field of medical sciences, health care economics and automobile insurance claims [1]. The GRM is used, when the dependent variable is positively skewed and assumes that the dependent variable has a gamma distribution [2]. The usual maximum likelihood estimator (MLE) is used to estimate the parameter vector $\beta$ of the GRM by means of iterative reweighed least squares (IRWLS) algorithm. The MLE minimizes the weighted sum of squared error. Though the MLE estimator cannot be suitable if it is suspected that $\beta$ may belong to a linear subspace such as $F\beta =f$, where f is an $m\times 1$ vector of known elements and F is the $m\times p$ known matrix of full rank with $m<p.$ In this condition, a restricted maximum likelihood estimator (RMLE) should be used [3,4]. The problem of multicollinearity and its consequences on a GLM are well known, for example, one of the most important consequences of multicollinearity on the restricted and unrestricted estimators in the GRM is that both estimators provide large variances of the regression coefficients. Another consequence of multicollinearity is the wider confidence interval, decreased statistical power which results in an increase in the probability of type II error in hypothesis testing in terms of the parameters [5].

Unrestricted ridge estimators under the classical linear regression model and the GRM have been considered respective by Hoerl and Kennard [6] and Amin et al. [7] to remedy the problem caused by multicollinearity. However, the regression coefficient estimates are obtained by using the restricted estimator when the prior information for regression coefficients is available, which be able to form linear restriction $F\beta =f$ in the GLM. For instance, the linear restriction exists in sum-to-zero parameterization, when the regressors are qualitative and in quantal response problems with constant relative potency between drugs (see for more details, 3). Therefore, in this paper, the model of interest is the GRM which relates to the system of linear restrictions. Nyquist [3] proposed the RMLE for the GLM under linear restrictions on the parameters. However, in the presence of multicollinearity, the weighted matrix of cross products is ill-conditioned, which leads to instability and high variances of both the MLE (unrestricted) and RMLE. To overcome the problem of multicollinearity, different kinds of shrinkage estimator have been proposed. For the linear regression model, Kaçiranlar et al. [8] developed the restricted Liu estimator by combing the Liu estimator and the restricted least square estimator (RSLE), and Sarkar [9] proposed restricted ridge regression approach by combining ordinary ridge regression and RSLE. However, literature on the restricted ridge regression under the GRM is limited.

The aim of this paper is to propose a new restricted gamma ridge regression estimator (RGRRE) by combining the GRRE and RMLE by following the work of Sarkar [9]. The motivation at the back of the new estimator is as follows. Under certain circumstances, a reparameterization in the forms of linear combinations of the regression coefficients is found when the linear restrictions on the regression coefficients are true. Now consider the problem of multicollinearity under the reparametrized model, then it is required to modify the RMLE in the line of GRRE to obtain the efficient estimates. The mean square error (MSE) properties of the RGRRE are studied and show the superiority of RGRRE to the RMLE. We also suggest some methods for estimating the value of ridge parameter.

The rest of this study is organized as follows: The GRM and existing estimators are discussed in Section 2. In Section 3, we derive the MSE properties of the proposed estimator and the theoretical comparison with existing estimators is considered. In Section 4, we suggest some estimators for the selection of the ridge parameter of the RGRRE. The design of the Monte Carlo simulation study and its results are provided in Section 5. A real-life dataset is analysed in Section 6. Some concluding remarks are presented in Section 7.

2. The gamma regression model and estimators

In the GLM framework, the response variable $y$ is assumed to follow an exponential family distribution with mean $\mu$. The random variables $y_i(i=1,\dots ,n)$ contain n independent observations, i.e. $y_1,y_2,\dots ,y_n$. Suppose that the probability density function of $y_i$ is shown as below $\begin{array}{r}f(y_i,{\theta }_i,\phi )=\exp \left[\frac{y_i{\theta }_i-b({\theta }_i)}{a(\phi )}+c(y_i,\phi )\right],\end{array}$where ${\theta }_i$ is the location parameter, $\phi$ is the dispersion parameter and $b({\theta }_i)$ is the cumulant function. The GLM employs the relation (1) $\begin{array}{r}\text{g}({\mu }_i)={\eta }_i=x_i^{\mathrm{\prime }}\beta ,i=1,\dots ,n\end{array}$(1) where $E(y_i)={\mu }_i=\frac{\mathrm{\partial }b({\theta }_i)}{\mathrm{\partial }{\theta }_i}$, $\text{g}(\cdot )$ is the monotonic differentiable log link function, ${\eta }_i$ is the linear predictor, $\beta =({\beta }_1,{\beta }_2,\dots ,{\beta }_p{)}^{\mathrm{\prime }}$ is a $p\times 1$ vector of regression coefficients and $x_i$ is the ith row of $X=[x_{i1},x_{i2},\dots ,x_{ip}{]}^{\mathrm{\prime }}$ which is an $n\times p$ matrix with $p$ non-stochastic explanatory variables. Then the log-likelihood function is defined as $\begin{array}{r}{\ell }_i=\left[\frac{y_i{\theta }_i-b({\theta }_i)}{a(\phi )}+c(y_i,\phi )\right]\end{array}$The MLE of ${\beta }_j$ is obtained by differentiating the function below $\begin{array}{r}\ell (\beta )=\sum _{i=1}^n\left[\frac{y_i{\theta }_i-b({\theta }_i)}{a(\phi )}+c(y_i,\phi )\right]\end{array}$which is $\begin{array}{r}\frac{\mathrm{\partial }\ell }{\mathrm{\partial }{\beta }_j}=\frac{1}{a(\phi )}\sum _{i=1}^n\left[\frac{y_i-{\mu }_i}{w_i}{g}^{\mathrm{\prime }}({\mu }_i)f_j(x_i)\right],j=1,2,\dots ,p.\end{array}$Consider the response variable $y_i$ contains n independent observations that comes from the gamma distribution. Let $y_i\sim Gamma(\mu ,\varphi )$ by using the reparameterization as $\alpha ={\varphi }^{-1}$ and $\beta =\mu \varphi$ where $\mu =E(y_i)$ is the mean function of the response variable and $\varphi >0$ is the dispersion parameter (see for more details of the GRM estimation [7,10]). The mean function of the GRM is defined as $E(y_i)=\frac{\mathrm{\partial }b({\theta }_i)}{\mathrm{\partial }{\theta }_i}={\mu }_i=exp(x_i^{\mathrm{\prime }}\beta )$ by using log link function $\text{g}({\mu }_i)={\eta }_i=x_i^{\mathrm{\prime }}\beta$ as defined in Equation (1). The most common estimation method for the GRM is the MLE. The MLE is obtained by solving $\begin{array}{r}\frac{\mathrm{\partial }l}{\mathrm{\partial }\beta }=\frac{dl}{d{\theta }_i}\frac{d{\theta }_i}{d{\mu }_i}\frac{d{\mu }_i}{d{\eta }_i}\frac{\mathrm{\partial }{\eta }_i}{\mathrm{\partial }\beta }=\sum _{i=1}^n\frac{(y_i-{\mu }_i)}{a(\phi )}\frac{1}{Va{r}_{\mu }}\frac{d{\mu }_i}{d{\eta }_i}{x}_i.\end{array}$Since the above function is non-linear in $\beta$, so we can solve the above expression iteratively through Fisher’s scoring method. Consider ${\beta }^{(m)}$ be the estimated value of the MLE of $\beta$ with iterations $t(t\ge 1)$ which may be written as $\begin{array}{r}{\beta }^{(t+1)}={\beta }^{(t)}+\{I({\beta }^{(t)}){\}}^{-1}S({\beta }^{(t)}),\end{array}$where $I({\beta }^{(t)})$ represents Fisher information matrix and $S(\beta )$ is the $p\times 1$ vector of scores, and both $S(\beta )$ and $I({\beta }^{(t)})$ are computed at the last iteration ${\beta }^{(t)}$, where the convergence is achieved. After the final iteration, the MLE can be obtained by the IRWLS method as (2) $\begin{array}{r}{\hat{\beta }}_{MLE}=(A{)}^{-1}{X}^{\mathrm{\prime }}\hat{W}z,\end{array}$(2) where $A=X^{\mathrm{\prime }}\hat{W}X$, $\hat{W}=diag\{(1/{\mu }_i^2){\mu }_i^2\}$ and $z$ is $n\times 1$ vector with elements $z_i=x_i^{{}^{\mathrm{\prime }}}\beta +\frac{(y_i-{\mu }_i)}{{\mu }_i^2}$ being evaluated at the last iteration ${\beta }^{(t)}$, whereas ${\mu }_i=exp(x_i^{{}^{\mathrm{\prime }}}\hat{\beta })$ represents the mean function of the response variable using log link function. Let $X_n^{\ast }={\hat{W}}_n^{1/2}X$ and $Z^{\ast }={\hat{W}}_n^{1/2}z$, then $\begin{array}{r}{\check{\beta }}_{MLE}=(X{{}_n^{\ast }}^{\mathrm{\prime }}{X}_n^{\ast }{)}^{-1}{X}_n^{\ast }{Z}^{\ast }.\end{array}$By following Kibria and Saleh [4], we assume two conditions (i) $\frac{1}{n}(X{{}_n^{\ast }}^{\mathrm{\prime }}{X}_n^{\ast })=A$, as $n\to \mathrm{\infty }$ and (ii) $\underset{1\le i\le n}{max}{x}_{n_i}^{\ast }(X{{}_n^{\ast }}^{\mathrm{\prime }}{X}_n^{\ast }{)}^{-1}{x}_{n_i}^{\ast }\to 0$ as $n\to \mathrm{\infty }$, where $A=X^{\mathrm{\prime }}\hat{W}X$ is finite and p.d matrix and $x_{n_i}^{\ast }$ is the ith row of the matrix $X_n^{\ast }$. The asymptotic distribution of ${\check{\beta }}_{MLE}$ is stated as $\begin{array}{r}{n}^{1/2}F({\check{\beta }}_{MLE}-\beta )\sim N_m[0,(F^{\mathrm{\prime }}{A}^{-1}F)].\end{array}$

2.1. The RMLE

One method to improve the efficiency of the estimators is the use of extraneous or prior information. In practice, such prior information might be accessible in relation to the regression coefficients. For instance, in applied economics, the constant returns to scale imply that the exponents in a Cobb–Douglas production function should sum to unity. In the second example, the absence of money illusion on the part of consumers implies that the sum of money income and price elasticities in a demand function should be zero. This type of prior information may be available from an extraneous source, user experience, some theoretical consideration, among others. To make use of such information in enhancing the estimation of regression coefficients, it can be said in the form of linear restrictions.

Our primary aim is to estimate $\beta =({\beta }_1,{\beta }_2,\dots ,{\beta }_p{)}^{\mathrm{\prime }}$ when it is suspected that $\beta$ belongs to the linear sub-space defined by (3) $\begin{array}{r}{F}_r^{{}^{\mathrm{\prime }}}\beta =f_r,r=1,2,\dots ,m,\end{array}$(3) where $f_r$ are scalars, $F_r$ are $p\times 1$ known vectors and $m$ linearly independent restriction on parameter vector $\beta$. In such situation, we may use restricted estimator of $\beta$ [3,4,11,12]. The RMLE [3] is obtained to maximize the log-likelihood function of the GRM over $\beta$ under restrictions $F_r^{\mathrm{\prime }}\beta =f_r$. One method for solving restricted optimization problems is the quadratic penalty function. The quadratic function for the RMLE is given as $\begin{array}{r}\mathrm{\Theta }(\beta ,\lambda )=\ell (\beta )-\frac{1}{2}\sum _{r=1}^m{\lambda }_r(f_r-F_r^{\mathrm{\prime }}\beta {)}^2\end{array}$and $ma{x}_{\beta }\mathrm{\Theta }(\beta ,\lambda )$ for fixed and positive value of ${\lambda }_j$. Differentiating $\mathrm{\Theta }(\beta ,\lambda )$ with respect to ${\beta }_j$ equals $\begin{array}{r}{\mathrm{P}}_j(\beta ,\lambda )=\sum _{i=1}^n\frac{(y_i-{\mu }_i)}{a(\phi )}\frac{1}{Va{r}_{\mu }}\frac{d{\mu }_i}{d{\eta }_i}{x}_{ij}-\frac{1}{2}\sum _{r=1}^m{F}_{rj}{\lambda }_r(f_j-F_j^{\mathrm{\prime }}\beta ).\end{array}$We compute the RMLE by using the method similar to that for unrestricted estimator. Therefore, the $(t+1)\text{th}$ approximation of the RMLE is finally obtained as (4) $\begin{array}{r}{\hat{\beta }}_{RMLE}={\hat{\beta }}_{MLE}+A^{-1}{F}^{\mathrm{\prime }}(F{A}^{-1}{F}^{\mathrm{\prime }}{)}^{-1}(f-F{\hat{\beta }}_{MLE}).\end{array}$(4) For testing the hypothesis $H_o:F\beta =f$, the Wald-type test is defined as $\begin{array}{r}{\mathcal{L}}_n=n{(F{\check{\beta }}_{MLE}-f)}^{\mathrm{\prime }}[F^{\mathrm{\prime }}{(n^{-1}{A}_n)}^{-1}F{]}^{-1}(F{\check{\beta }}_{MLE}-f).\end{array}$Under $H_o:F\beta =f$, ${\mathcal{L}}_n\stackrel{D}{\to }{\chi }_m^2$ as $n\to \mathrm{\infty }$, ${\mathcal{L}}_n$ follows central ${\chi }^2$ distribution with m degrees of freedom.

Clearly, $E({\hat{\beta }}_{RMLE})\ne \beta$ unless (3) holds: $\begin{array}{r}Bias({\hat{\beta }}_{RMLE})=E({\hat{\beta }}_{RMLE})-\beta =A^{-1}{F}^{\mathrm{\prime }}(F{A}^{-1}{F}^{\mathrm{\prime }}{)}^{-1}(f-F\beta ).\end{array}$The variance–covariance matrix of ${\hat{\beta }}_{RMLE}$ is defined as $\begin{array}{r}Cov({\hat{\beta }}_{RMLE})=\phi [A^{-1}-A^{-1}{F}^{\mathrm{\prime }}{(F{A}^{-1}{F}^{\mathrm{\prime }})}^{-1}F{A}^{-1}],\end{array}$where $\phi$ is the estimated dispersion parameter computed as $\phi =\frac{1}{n-p}\sum _{i=1}^n\frac{{(y_i-{\hat{\mu }}_i)}^2}{{\hat{\mu }}_i^2}.$

The performance of the RMLE is superior to the MLE since RMLE has a smaller sampling variance than the MLE. The asymptotic variance–covariance matrix of ${\hat{\beta }}_{MLE}$ is defined as $Cov({\hat{\beta }}_{MLE})=\phi (A^{-1})$. Therefore, it is shown that $\begin{array}{r}[Cov({\hat{\beta }}_{MLE})-Cov({\hat{\beta }}_{RMLE})]=\phi (A^{-1})-\phi [A^{-1}-A^{-1}{F}^{\mathrm{\prime }}{(F{A}^{-1}{F}^{\mathrm{\prime }})}^{-1}F{A}^{-1}]\end{array}$which is a positive semidefinite (psd) matrix. So, we conclude that the RMLE has minimum sampling variance as compared to the MLE. Amin et al. [7] proposed the GRRE for the GRM (5) $\begin{array}{r}\hat{\beta }(k)=A_k{\hat{\beta }}_{MLE},k\ge 0\end{array}$(5) where $A_k=(I_p+k{A}^{-1}{)}^{-1}$, $I_p$ is an identity matrix of order $p\times p$ and $k\ge 0$ is the ridge parameter. The MSE properties of the $\hat{\beta }(k)$ are defined by Amin et al. [7].

3. Proposed estimator

The RMLE of $\beta$ is obtained by maximizing the log-likelihood function of the GRM subject to the restrictions given in Equation (3). Now considering the problem of multicollinearity under the reparametrized model, the method of RMLE may produce poor estimates and provide misleading information as in MLE in the presence of multicollinearity. Therefore, it is required to modify the RMLE in the line of GRRE to obtain efficient estimates under a set of linear restrictions by following the work of Sarkar [9]. Quadratic function for the RGRRE is defined as $\begin{array}{r}\mathrm{\Theta }(\beta ,k,\lambda )={\beta }^{\mathrm{\prime }}\beta -\frac{2}{k}l(\beta )+\sum _{r=1}^m{\lambda }_r(f_r-F_r^{{}^{\mathrm{\prime }}}\beta {)}^2,\end{array}$where $1/k$ is the Lagrange multipliers. The quadratic function $\mathrm{\Theta }(\beta ,k,\lambda )$ combines objective functions of GRRE [13] and RMLE [3], and the same idea is also carried out by Kurtoǧlu and Özkale [14] for the count regression model where the dispersion parameter equals to 1. Differentiating $\mathrm{\Theta }(\beta ,k,\lambda )$ with respect to ${\beta }_j$ equals $\begin{array}{r}{\mathrm{P}}_j(\beta ,k,\lambda )=2\left[{\beta }_j-\frac{1}{k}\sum _{i=1}^n\frac{(y_i-{\mu }_i)}{a(\phi )}\frac{1}{Va{r}_{\mu }}\frac{d{\mu }_i}{d{\eta }_i}{x}_{ij}+\sum _{r=1}^m{F}_{rj}{\lambda }_r(f_j-F_j^{\mathrm{\prime }}\beta )\right].\end{array}$Now define $\mathcal{H}(\beta ,k,\lambda )$ is $p\times p$ matrix with elements ${\mathrm{\Re }}_{jq}(\beta ,k,\lambda )$ and taking the second-order derivates of ${\mathrm{P}}_j(\beta ,k,\lambda )$ as $\begin{array}{r}{\mathrm{\Re }}_{jq}(\beta ,k,\lambda )=\frac{{\mathrm{\partial }}^2(\mathrm{\Theta }(\beta ,k,\lambda ))}{\mathrm{\partial }{\beta }_j\mathrm{\partial }{\beta }_q}=2\left[{\delta }_{jq}-\frac{1}{k}\frac{{\mathrm{\partial }}^2(\ell (\beta ))}{\mathrm{\partial }{\beta }_j\mathrm{\partial }{\beta }_q}+\sum _{r=1}^m{\lambda }_r{F}_{rj}{F}_{rq}\right].\end{array}$Taking the expectation of both sides, we have $\begin{array}{rl}E\left[\frac{{\mathrm{\partial }}^2(\mathrm{\Theta }(\beta ,k,\lambda ))}{\mathrm{\partial }{\beta }_j\mathrm{\partial }{\beta }_q}\right]& =-2\left[{\delta }_{jq}-\frac{1}{k}E\left\{-\frac{{\mathrm{\partial }}^2(\ell (\beta ))}{\mathrm{\partial }{\beta }_j\mathrm{\partial }{\beta }_q}\right\}+\sum _{r=1}^m{\lambda }_r{F}_{rj}{F}_{rq}\right]\\ & =-2\left[{\delta }_{jq}+\frac{1}{k}\sum _{i=1}^n\frac{x_{ij}{x}_{iq}}{a(\phi )}\frac{1}{{\mu }_i^2}{\left(\frac{d{\mu }_i}{d{\eta }_i}\right)}^2+\sum _{r=1}^m{\lambda }_r{F}_{rj}{F}_{rq}\right],\end{array}$where ${\delta }_{jq}=\{\begin{array}{ll}1& ifj=q\\ 0& otherwise\end{array}$.

Let ${\beta }_F(k)$ is the RGRRE of $\beta$ with iterations $t(t\ge 1)$. Subsequently, by means of the Fisher scoring method this case yields $\begin{array}{r}{\beta }_F(k{)}^{(t+1)}={\beta }_F(k{)}^{(t)}+\mathcal{H}({{\beta }_F}^{(t)},k,\lambda {)}^{-1}\mathrm{\Theta }({{\beta }_F}^{(t)},k,\lambda ),\end{array}$Based on the objective functions of RMLE $\mathrm{\Theta }(\beta ,\lambda )$ and GRRE $\mathrm{\Theta }(\beta ,k,\lambda )$, the RGRRE is defined as (6) $\begin{array}{r}{\hat{\beta }}_F(k)=\hat{\beta }(k)+(A+k{I}_p{)}^{-1}{F}^{\mathrm{\prime }}[F{(A+k{I}_p)}^{-1}{F}^{\mathrm{\prime }}{]}^{-1}(f-F{\hat{\beta }}_{MLE}),k>0.\end{array}$(6) The final form of the restricted ridge estimator in the GLM was proposed by Kurtoğlu and Özkale [14] and it is same for the GRM. However, the MSE properties of GRM are different as compared to the count regression model by considering the effect of dispersion parameter in the estimation methods of optimal shrinkage parameter $k$ in the RGRRE. The performance of the shrinkage estimators is different for different forms of the GLM in the restricted estimators (e.g. [4,10–12]).

The RGRRE can be simplified as (6a) $\begin{array}{r}{\hat{\beta }}_F(k)=A_k{\hat{\beta }}_{RMLE},\end{array}$(6a) where $A_k=(I_p+k{A}^{-1}{)}^{-1}$ and ${\hat{\beta }}_{RMLE}={\hat{\beta }}_{MLE}+A^{-1}{F}^{\mathrm{\prime }}(F{A}^{-1}{F}^{\mathrm{\prime }}{)}^{-1}(f-F{\hat{\beta }}_{MLE})$. It can be easily seen that ${\hat{\beta }}_F(k)={\hat{\beta }}_{RMLE}$ when $k=0$.

We compute the bias of ${\hat{\beta }}_F(k)$ by using (6) as $\begin{array}{rl}Bias({\hat{\beta }}_F(k))& =E[{\hat{\beta }}_F(k)]-\beta \\ & =[A_k\beta +A_k{A}^{-1}{F}^{\mathrm{\prime }}{(F{A}^{-1}{F}^{\mathrm{\prime }})}^{-1}(f-F\beta )]-\beta \\ & =(A_k-I_p)\beta +A_k{A}^{-1}{F}^{\mathrm{\prime }}(F{A}^{-1}{F}^{\mathrm{\prime }}{)}^{-1}(f-F\beta )\\ & =-k(A+k{I}_p{)}^{-1}\beta +A_k{A}^{-1}{F}^{\mathrm{\prime }}(F{A}^{-1}{F}^{\mathrm{\prime }}{)}^{-1}(f-F\beta )\end{array}$So the RGRRE is a biased estimator of parameter vector $\beta$ unless $k=0$ and $f-F\beta =0$, where $0$ is the $m\times 1$ null vector.

The variance–covariance matrix of ${\hat{\beta }}_F(k)$ is computed as $\begin{array}{r}Cov({\hat{\beta }}_F(k))=\phi [A_k\{A^{-1}-A^{-1}{F}^{\mathrm{\prime }}{(F{A}^{-1}{F}^{\mathrm{\prime }})}^{-1}F{A}^{-1}\}{A_k}^{{}^{\mathrm{\prime }}}].\end{array}$

3.1. MSE properties and superiority of the ${\hat{\mathit{\beta }}}_{\mathit{F}}(\mathit{k})$

In this section, we define the MSE properties and gauge the performance of ${\hat{\beta }}_F(k)$. The matrix means squared error (MMSE) of an estimator $\hat{\upsilon }$ of $\upsilon$ can be defined as (7) $\begin{array}{r}MMSE(\hat{\upsilon })=E(\hat{\upsilon }-\upsilon {)}^{{}^{\mathrm{\prime }}}(\hat{\upsilon }-\upsilon )=Cov(\hat{\upsilon })+Bias(\hat{\upsilon })Bias(\hat{\upsilon }{)}^{{}^{\mathrm{\prime }}},\end{array}$(7) where $Cov(\hat{\upsilon })$ denotes the variance–covariance matrix of an estimator $\hat{\upsilon }$ and $Bias(\hat{\upsilon })=E(\hat{\upsilon })-\upsilon$ represents the bias vector. The scalar MSE (SMSE) of $\hat{\upsilon }$ can be found by applying the trace operator which is given by (8) $\begin{array}{r}SMSE(\hat{\upsilon })=tr[MMSE(\hat{\upsilon })]=tr[Cov(\hat{\upsilon })]+Bias(\hat{\upsilon }{)}^{{}^{\mathrm{\prime }}}Bias(\hat{\upsilon }).\end{array}$(8) Let ${\hat{\upsilon }}_j(j=1,2)$ be two competitive estimators of parameter $\upsilon$, and the estimator ${\hat{\upsilon }}_2$ is said to be superior to ${\hat{\upsilon }}_1$ in the form of MMSE criterion if $MMSE({\hat{\upsilon }}_2)-MMSE({\hat{\upsilon }}_1)\ge 0$. Moreover, if the estimator ${\hat{\upsilon }}_2$ dominates the estimator ${\hat{\upsilon }}_1$ in the form of MMSE criterion, then $SMSE({\hat{\upsilon }}_1)\ge SMSE({\hat{\upsilon }}_2)$. For the comparison of the following discussion, we list the following lemmas for making the comparison of the estimators.

Lemma 1.

Let $\mathbb{A}$ is a real symmetric matrix, $\mathbb{P}$ is a matrix, then $\mathbb{A}\ge 0\iff \mathrm{\forall }\mathbb{P},{\mathbb{P}}^{\mathrm{\prime }}\mathbb{A}\mathbb{P}\ge 0\iff$ each eigenvalue of matrix $\mathbb{A}$ is nonnegative.

Proof:

See Wang et al. [15]

Lemma 2.

Suppose that matrices $\mathbb{A}$ and $\mathbb{B}$ are not singular, and $\mathbb{C}and\mathbb{D}$ are matrices with proper orders, then $(\mathbb{A}+\mathbb{C}\mathbb{B}\mathbb{D}{)}^{-1}={\mathbb{A}}^{-1}-{\mathbb{A}}^{-1}\mathbb{C}({\mathbb{B}}^{-1}+\mathbb{D}{\mathbb{A}}^{-1}\mathbb{C}{)}^{-1}\mathbb{D}{\mathbb{A}}^{-1}.$

Proof:

See Rao et al. [16]

3.1.1. Comparison between ${\hat{\mathit{\beta }}}_{\mathit{F}}(\mathit{k})$ and ${\hat{\mathit{\beta }}}_{\mathit{R}\mathit{M}\mathit{L}\mathit{E}}$ when the prior restrictions on the parameters are true, i.e. $\mathit{\tau }=(\mathit{f}-\mathit{F}\mathit{\beta })=0$

It is evident that $E({\hat{\beta }}_{RMLE})\ne \beta$ unless (3) holds. The ${\hat{\beta }}_{RMLE}$ is an unbiased estimator when $\tau =0$ while ${\hat{\beta }}_F(k)$ is always a biased estimation method.

Theorem 3.1.

Under the GRM, the ${\hat{\beta }}_F(k)$ is a biased estimator and ${\hat{\beta }}_{RMLE}$ is an unbiased estimator when $\tau =0$. Though, the $Cov({\hat{\beta }}_F(k))\le Cov({\hat{\beta }}_{RMLE})$ for $k\ge 0$.

Proof:

As regards performance by the variance–covariance matrices of ${\hat{\beta }}_F(k)$ and ${\hat{\beta }}_{RMLE}$, we compute the difference of two variance–covariance matrices as (9) $\begin{array}{rl}& [Cov({\hat{\beta }}_{RMLE})-Cov({\hat{\beta }}_F(k))]\\ & =\phi [A^{-1}-A^{-1}{F}^{\mathrm{\prime }}{(F{A}^{-1}{F}^{\mathrm{\prime }})}^{-1}F{A}^{-1}]-\phi [A_k\{A^{-1}-A^{-1}{F}^{\mathrm{\prime }}{(F{A}^{-1}{F}^{\mathrm{\prime }})}^{-1}F{A}^{-1}\}{A_k}^{{}^{\mathrm{\prime }}}]\\ & =[\phi A_k\{k^2{A}^{-1}G{A}^{-1}+kG{A}^{-1}+k{A}^{-1}G\}{A_k}^{{}^{\mathrm{\prime }}}],\end{array}$(9) where $G=A^{-1}-A^{-1}{F}^{\mathrm{\prime }}(F{A}^{-1}{F}^{\mathrm{\prime }}{)}^{-1}F{A}^{-1}$ ($G=$ psd), $A=X^{\mathrm{\prime }}\hat{W}X$ ($A=$ positive definite) and $A_k=(I_p+k{A}^{-1}{)}^{-1}$. From (9), we conclude that $A^{-1}G$ is a non-negative definite (nnd) matrix and $A^{-1}G{A}^{-1}$ is a psd matrix. Therefore, $[Cov({\hat{\beta }}_{RMLE})-Cov({\hat{\beta }}_F(k))]$ is a psd for $k\ge 0$. Hence, the sampling variance of ${\hat{\beta }}_F(k)$ has smaller than the ${\hat{\beta }}_{RMLE}$.

Now, we discuss the SMSE properties of the ${\hat{\beta }}_F(k)$ and show the comparison of ${\hat{\beta }}_F(k)$ to the ${\hat{\beta }}_{RMLE}$. Following [11] and [3], we define the SMSE of ${\hat{\beta }}_{RMLE}$ for the GRM as below (10) $\begin{array}{r}SMSE({\hat{\beta }}_{RMLE})=\phi tr(G)=\phi \sum _{j=1}^p{m}_{jj},\end{array}$(10) where $m_{jj}\ge 0$ represents the jth diagonal element of the matrix $M=Q^{\mathrm{\prime }}GQ$, Q is the orthogonal matrix such that $Q^{\mathrm{\prime }}GQ=\mathrm{\Lambda }=diag({\lambda }_1,{\lambda }_2,\dots ,{\lambda }_p)$. The SMSE of the RGRRE is computed as (11) $\begin{array}{r}SMSE({\hat{\beta }}_F(k))=\phi tr[Cov({\hat{\beta }}_F(k))]+[Bias({\hat{\beta }}_F(k)){]}^{{}^{\mathrm{\prime }}}[Bias({\hat{\beta }}_F(k))]\end{array}$(11)

If we assume that the prior restrictions hold, i.e.$\tau =0$, then (11) may be written as $\begin{array}{r}SMSE({\hat{\beta }}_F(k))=\phi tr(A_{k}G{A}_k^{\mathrm{\prime }})+k^2{\beta }^{\mathrm{\prime }}(A+k{I}_p{)}^{-2}\beta .\end{array}$After simplification, the SMSE of the RGRRE is defined as (12) $\begin{array}{r}\text{S}MSE({\hat{\beta }}_F(k))=\phi \sum _{j=1}^p\frac{{\lambda }_j^2}{{({\lambda }_j+k)}^2}{m}_{jj}+k^2\sum _{j=1}^p\frac{{\alpha }_j^2}{{({\lambda }_j+k)}^2}={\gamma }_1(F_k)+{\gamma }_2(F_k)\end{array}$(12) where $\alpha$ is the jth element of $Q^{\mathrm{\prime }}\beta$, ${\lambda }_j$ is the jth eigenvalue of the matrix A, ${\gamma }_1(F_k)\text{and}{\gamma }_2(F_k)$ represent the total variance and squared bias of ${\hat{\beta }}_F(k)$ respectively. From (12), it can be noted that the bias of the RGRRE is the same as the bias of the GRRE when the prior restrictions on the parameters are true, i.e. $\tau =(f-F\beta )=0$. By following [6], we show the following SMSE properties of ${\hat{\beta }}_F(k)$ for the GRM.

Theorem 3.2.

The total variance ${\gamma }_1(F_k)$ is a continuous and monotonically decreasing function of k.

Proof:

Differentiating the expression ${\gamma }_1(F_k)$ with respect to k as below (13) $\begin{array}{r}\frac{\mathrm{\partial }\{{\gamma }_1(F_k)\}}{\mathrm{\partial }k}=-2\phi \sum _{j=1}^p\frac{{\lambda }_j^2}{{({\lambda }_j+k)}^3}{m}_{jj}\end{array}$(13) Based on (13), it is evident that ${\gamma }_1(F_k)$ is a continuous and monotonically decreasing function of k since $\frac{\mathrm{\partial }\{{\gamma }_1(F_k)\}}{\mathrm{\partial }k}\to -\mathrm{\infty }$ when $k\to 0+$ and ${\lambda }_p\to 0$.

Theorem 3.3.

The squared bias ${\gamma }_2(F_k)$ is a continuous and monotonically increasing function of k.

Proof:

Differentiating the expression ${\gamma }_2(F_k)$ with respect to k as below (14) $\begin{array}{r}\frac{\mathrm{\partial }\{{\gamma }_2(F_k)\}}{\mathrm{\partial }k}=2k\sum _{j=1}^p\frac{{\lambda }_j{\alpha }_j^2}{{({\lambda }_j+k)}^3}.\end{array}$(14) Equation (14) indicates that ${\gamma }_2(F_k)$ is a continuous and monotonically increasing function of k for $k>0$ and ${\lambda }_j>0$.

Theorem 3.4.

Under the GRM, there always exists $k>0$ in the range $0<k<\phi /\left[max\left(\frac{{\alpha }_j^2}{({\lambda }_j{m}_{jj})}\right)\right]$ such that $\text{S}MSE({\hat{\beta }}_F(k))<SMSE({\hat{\beta }}_{RMLE})$ when $\tau =0$.

Proof:

The first derivative of (12) with respect to k equals (15) $\begin{array}{r}\frac{\mathrm{\partial }\{\text{S}MSE({\hat{\beta }}_F(k))\}}{\mathrm{\partial }k}=-2\phi \sum _{j=1}^p\frac{{\lambda }_j^2}{{({\lambda }_j+k)}^3}{m}_{jj}+2k\sum _{j=1}^p\frac{{\lambda }_j{\alpha }_j^2}{{({\lambda }_j+k)}^3}=2\sum _{j=1}^p\frac{k{\lambda }_j{\alpha }_j^2-\phi {\lambda }_j^2{m}_{jj}}{{({\lambda }_j+k)}^3}.\end{array}$(15) As discussed earlier $m_{jj}\ge 0$ and ${\lambda }_j>0$, $\mathrm{\forall }j=1,2,\dots ,p$. It is well known that total variance and total bias are monotonically decreasing and increasing function of k. It is then evident that $\frac{\mathrm{\partial }\{{\gamma }_1(F_k)\}}{\mathrm{\partial }k}$ and $\frac{\mathrm{\partial }\{{\gamma }_2(F_k)\}}{\mathrm{\partial }k}$ are always non-positive and non-negative, respectively. Thus to prove the theorem it is enough to condition that $0<k<\phi /\left[max\left(\frac{{\alpha }_j^2}{({\lambda }_j{m}_{jj})}\right)\right]$ for $\frac{\mathrm{\partial }\{\text{S}MSE({\hat{\beta }}_F(k))\}}{\mathrm{\partial }k}$ to be negative. Hence, we have proven that the existence of a value for $k,$ which showed the superiority of RGRRE over the GRRE.

Remark.

From Theorem 3.4, it is noted that $\text{M}=Q^{\mathrm{\prime }}GQ={\mathrm{\Lambda }}^{-1}-B$, where $B=Q^{\mathrm{\prime }}{A}^{-1}{F}^{\mathrm{\prime }}(F{A}^{-1}{F}^{\mathrm{\prime }}{)}^{-1}F{A}^{-1}Q$ and $\text{M}$ is a psd matrix. Consequently, $m_{jj}=\frac{1}{{\lambda }_j}-b_{jj}\le \frac{1}{{\lambda }_j}$ and $b_{jj}=diag(B)$. It can also be noted that $\phi /\left[max\left(\frac{{\alpha }_j^2}{({\lambda }_j{m}_{jj})}\right)\right]\le \phi /{\alpha }_{max}^2$. Therefore, we conclude that one of the beneficial consequences of involving exact prior information is that the range of values of k for the dominance of the RGRRE over the RMLE becomes reduced as compared to that for the dominance of GRRE over the traditional MLE in the sense of SMSE criterion.

3.1.2. Comparison between ${\hat{\mathit{\beta }}}_{\mathit{F}}(\mathit{k})$ and $\hat{\mathit{\beta }}(\mathit{k})$ when the prior restrictions on the parameters are true, i.e. $\mathit{\tau }=(\mathit{f}-\mathit{F}\mathit{\beta })=0$

This section compares the performance of ${\hat{\beta }}_F(k)$ and $\hat{\beta }(k)$.

Theorem 3.5.

Under the GRM, the $Cov({\hat{\beta }}_F(k))$ is less than $Cov(\hat{\beta }(k))$ for $k\ge 0\text{iff}\tau =0$.

Proof:

Both ${\hat{\beta }}_F(k)$ and $\hat{\beta }(k)$ have same bias iff $\tau =0$, which can be defined as $\begin{array}{r}Bias({\hat{\beta }}_F(k))=Bias(\hat{\beta }(k))=-k(A+k{I}_p{)}^{-1}\beta .\end{array}$Thus we only make a comparison between the dispersion matrices of the ${\hat{\beta }}_F(k)$ and $\hat{\beta }(k)$. (16) $\begin{array}{rl}& [Cov(\hat{\beta }(k))-Cov({\hat{\beta }}_F(k))]\\ & =\phi [\mathcal{W}{(k)}^{-1}A\mathcal{W}{(k)}^{-1}]-\phi [A_k\{A^{-1}-A^{-1}{F}^{\mathrm{\prime }}{(F{A}^{-1}{F}^{\mathrm{\prime }})}^{-1}F{A}^{-1}\}{A}_k]\\ & =\phi [\mathcal{W}{(k)}^{-1}{A}^{-1}{F}^{\mathrm{\prime }}{(F{A}^{-1}{F}^{\mathrm{\prime }})}^{-1}F{A}^{-1}\mathcal{W}{(k)}^{-1}],\end{array}$(16) where $\mathcal{W}(k)=(A+k{I}_p)$, $A_k=\mathcal{W}(k)A$ and $\mathcal{W}(k)=A_k{A}^{-1}$. $[Cov(\hat{\beta }(k))-Cov({\hat{\beta }}_F(k))]$ is a psd matrix for $k\ge 0$. It is enough to show that RGRRE is superior to the GRRE.

3.1.3. Comparison between ${\hat{\mathit{\beta }}}_{\mathit{F}}(\mathit{k})$ and ${\hat{\mathit{\beta }}}_{\mathit{R}\mathit{M}\mathit{L}\mathit{E}}$ when $\mathit{\tau }\ne 0$

The performance of the ${\hat{\beta }}_F(k)$ ultimately depends on the assumed restrictions and its bias when the prior constraints on the parameters are not true. To make a comparison of ${\hat{\beta }}_F(k)$ and ${\hat{\beta }}_{RMLE}$ in the sense of SMSE, we illustrate Theorem 3.6.

Theorem 3.6.

Under the GRM with collinear regressors, if we have $\check{k}>0$, where $\check{k}=[min(\varphi {\lambda }_j^2{m}_{jj}+\breve{\tau }{}_j^{\ast }+{\alpha }_j{\lambda }_j\breve{\tau }{}_j^{\ast })]/[max({\lambda }_j{\alpha }_j^2+{\alpha }_j\breve{\tau }{}_j^{\ast })]$. Then $\text{S}MSE({\hat{\beta }}_F(k))<\text{S}MSE({\hat{\beta }}_{RMLE})$ for $0<k<\check{k}<\mathrm{\infty }$ when $\tau \ne 0$.

Proof:

Rewrite the SMSE of ${\hat{\beta }}_F(k)$ and ${\hat{\beta }}_{RMLE}$ when the restriction does not hold, respectively: $\begin{array}{r}\text{SMSE}({\hat{\beta }}_{RMLE})=\phi tr[G]+{{\tau }^{\ast }}^{\mathrm{\prime }}{A}^{-2}{\tau }^{\ast },\end{array}$where ${\tau }^{\ast }=F^{\mathrm{\prime }}(F{A}^{-1}{F}^{\mathrm{\prime }}{)}^{-1}\tau$. The final form of SMSE $({\hat{\beta }}_{RMLE})$ is defined as below: (17) $\begin{array}{r}\text{SMSE}\left({\hat{\beta }}_{RMLE}\right)=\sum _{j=1}^p\frac{\phi {\lambda }_j^2{m}_{jj}+\breve{\tau }{{}_j^{\ast }}^2}{{\lambda }_j^2},\end{array}$(17) where ${\hat{\tau }}^{\ast }:=Q^{\mathrm{\prime }\ast }$. The SMSE of ${\hat{\beta }}_F(k)$ may rewrite as below (18) $\begin{array}{rl}\text{SMSE}\left({\hat{\beta }}_F\left(k\right)\right)& =\phi tr\left[A_{k}G{A_k}^{\mathrm{\prime }}\right]+{\left[\mathcal{W}{\left(k\right)}^{-1}{\tau }^{\ast }-k\mathcal{W}{\left(k\right)}^{-1}\beta {{\tau }^{\ast }}^{\mathrm{\prime }}{A}^{-2}{\tau }^{\ast }\right]}^{\mathrm{\prime }}\\ & \times \left[\mathcal{W}{\left(k\right)}^{-1}{\tau }^{\ast }-k\mathcal{W}{\left(k\right)}^{-1}\beta {{\tau }^{\ast }}^{\mathrm{\prime }}{A}^{-2}{\tau }^{\ast }\right]\\ \delta \left(k\right)& =\text{SMSE}\left({\hat{\beta }}_F\left(k\right)\right)=\phi \sum _{j=1}^p\frac{{\lambda }_j^2{m}_{jj}}{{\left({\lambda }_j+k\right)}^2}+\sum _{j=1}^p\frac{k^2{\alpha }_j^2+\breve{\tau }{{}_j^{\ast }}^2-2k{\alpha }_j{\breve{\tau }}_j^{\ast }}{{\left({\lambda }_j+k\right)}^2}.\end{array}$(18) Taking the first derivative of (18) with respect to k to complete the theorem. $\begin{array}{rl}& \delta {\left(k\right)}^{\mathrm{\prime }}=\frac{\mathrm{\partial }\left\{\text{SMSE}\left({\hat{\beta }}_F\left(k\right)\right)\right\}}{\mathrm{\partial }k}=-2\phi \sum _{j=1}^p\frac{{\lambda }_j^2{m}_{jj}}{{\left({\lambda }_j+k\right)}^3}+2k\\ & \sum _{j=1}^p\frac{k{\lambda }_j{\alpha }_j^2+k{\alpha }_j{\breve{\tau }}_j^{\ast }-\breve{\tau }{{}_j^{\ast }}^2-{\alpha }_j{\lambda }_j{\breve{\tau }}_j^{\text{*}}}{{\left({\lambda }_j+k\right)}^3}\\ & =2\sum _{j=1}^p\frac{k{\lambda }_j{\alpha }_j^2+k{\alpha }_j{\breve{\tau }}_j^{\text{*}}-\left(\breve{\tau }{{}_j^{\text{*}}}^2+{\alpha }_j{\lambda }_j{\breve{\tau }}_j^{\text{*}}+\phi {\lambda }_j^2{m}_{jj}\right)}{{\left({\lambda }_j+k\right)}^3}.\end{array}$and so $\begin{array}{r}\delta {\left(0\right)}^{\mathrm{\prime }}=2\sum _{j=1}^p\frac{-\left(\dot{\tau }{}_j^{\ast 2}+{\alpha }_j{\lambda }_j{\breve{\tau }}_j^{\text{*}}+\phi {\lambda }_j^2{m}_{jj}\right)}{{\left({\lambda }_j\right)}^3}.\end{array}$Additionally, $\begin{array}{r}\delta \left(0\right)=\sum _{j=1}^p\frac{\phi {\lambda }_j^2{m}_j+{\dot{\tau }}_j^{\ast 2}}{{\lambda }_j^2}=\mathrm{SMSE}\left({\hat{\beta }}_{RMLE}\right).\end{array}$Thus, to prove the theorem, it is enough to show that $\delta (k{)}^{{}^{\mathrm{\prime }}}$ is negative. If $\delta (0{)}^{{}^{\mathrm{\prime }}}<0$, then for $0<k<\check{k}<\mathrm{\infty }$, where $\check{k}=\frac{[min(\varphi {\lambda }_j^2{m}_{jj}+\breve{\tau }{}_j^{\ast }+{\alpha }_j{\lambda }_j\breve{\tau }{}_j^{\ast })]}{[max({\lambda }_j{\alpha }_j^2+{\alpha }_j\breve{\tau }{}_j^{\ast })]}$, we have $\delta (k)<\delta (0)$ or $\text{S}MSE({\hat{\beta }}_F(k))<\text{S}MSE({\hat{\beta }}_{RMLE})$.

3.1.4. Comparison between ${\hat{\mathit{\beta }}}_{\mathit{F}}(\mathit{k})$ and $\hat{\mathit{\beta }}(\mathit{k})$ when $\mathit{\tau }\ne 0$

As we discussed the performance of ${\hat{\beta }}_F(k)$ in the previous section, so it is easy to show the superiority of ${\hat{\beta }}_F(k)$ to the $\hat{\beta }(k)$.

Theorem 3.7.

Under the GRM with collinear regressors when $\tau \ne 0$, if we have $k>0$, then $\text{S}MSE({\hat{\beta }}_F(k))<\text{S}MSE(\hat{\beta }(k))$ for $0<\frac{[max(\breve{\tau }{{}_j^{\ast }}^2-\varphi {\lambda }_j(1+{\lambda }_j{m}_{jj}))]}{[min(2{\alpha }_j\breve{\tau }{}_j^{\ast })]}<k<\mathrm{\infty }$.

Proof:

The SMSE of $\hat{\beta }(k)$ is defined as (19) $\begin{array}{r}\text{S}MSE(\hat{\beta }(k))=\phi \sum _{j=1}^p\frac{{\lambda }_j}{{({\lambda }_j+k)}^2}+k^2\sum _{j=1}^p\frac{{\alpha }_j^2}{{({\lambda }_j+k)}^2}\end{array}$(19) The SMSE difference is computed by using (18)–(19), $\begin{array}{r}\text{SMSE}(\hat{\beta }(k))-\text{SMSE}({\hat{\beta }}_F(k))=\sum _{j=1}^p\frac{2k{\alpha }_j\breve{\tau }{}_j^{\ast }-[\breve{\tau }{{}_j^{\ast }}^2-\varphi {\lambda }_j(1+{\lambda }_j{m}_{jj})]}{[min(2{\alpha }_j\breve{\tau }{}_j^{\ast })]}.\end{array}$The expression $2k{\alpha }_j\breve{\tau }{}_j^{\ast }-[\breve{\tau }{{}_j^{\ast }}^2-\varphi {\lambda }_j(1+m_{jj})]>0$ if $k>\frac{[max(\breve{\tau }{{}_j^{\ast }}^2-\varphi {\lambda }_j(1+{\lambda }_j{m}_{jj}))]}{[min(2{\alpha }_j\breve{\tau }{}_j^{\ast })]}>0$. This completes the theorem.

4. Estimation of the shrinkage parameter k

In practice, it is best to estimate the shrinkage parameter k in order to attain minimum MSE of the RMLE and RGRRE instead of MLE and GRRE. For this purpose, we find the optimal value of k by differentiating (12) with respect to k and equating to zero, we have (20) $\begin{array}{r}\frac{\mathrm{\partial }[MSE({\hat{\beta }}_F(k))]}{\mathrm{\partial }k}=\phi \sum _{j=1}^p\frac{{\lambda }_j^2}{{({\lambda }_j+k)}^3}{m}_{jj}-k\sum _{j=1}^p\frac{{\alpha }_j^2}{{({\lambda }_j+k)}^2}+k^2\sum _{j=1}^p\frac{{\alpha }_j^2}{{({\lambda }_j+k)}^2}=0\end{array}$(20) It is easy to see that the above equation may be simplified as (21) $\begin{array}{r}\frac{\mathrm{\partial }[MSE({\hat{\beta }}_F(k))]}{\mathrm{\partial }k}=\phi {\lambda }_j^2{m}_{jj}-k{\alpha }_j^2{\lambda }_j=0\end{array}$(21) By solving (21) for k, we have (22) $\begin{array}{r}{k}_j=\frac{\phi {\lambda }_j{m}_{jj}}{{\alpha }_j^2},\end{array}$(22) where ${\alpha }_j^2$ which is squared of ${\alpha }_j$ represents the jth element of the vector $\alpha =Q^{\mathrm{\prime }}\beta$. The details of these matrices are already discussed in the previous section. After estimating the optimal value of the shrinkage parameter, we propose several estimators to assess efficiency. Motivated by the work [17,18] where different ridge estimation methods have been proposed for estimating the shrinkage parameter. Following are our proposed ridge estimators for the RGRRE: $\begin{array}{rl}{k}_1& ={\hat{k}}_{mean}=\frac{{\sum }_{j=1}^p\left(\sqrt{\frac{\hat{\phi }{\lambda }_j{m}_{jj}}{{\hat{\alpha }}_j^2}}\right)}{p};\\ k_2& ={\hat{k}}_{median}=median\left(\frac{\hat{\phi }{\lambda }_j{m}_{jj}}{{\hat{\alpha }}_j^2}\right)\\ k_3& ={\hat{k}}_{max}=max\left(\sqrt{\frac{\hat{\phi }{\lambda }_j{m}_{jj}}{{\hat{\alpha }}_j^2}}\right);\\ k_4& ={\hat{k}}_{min}=min\left(\frac{\hat{\phi }{\lambda }_j{m}_{jj}}{{\hat{\alpha }}_j^2}\right)\\ k_5& ={\hat{k}}_{HM}=\frac{p}{{\sum }_{j=1}^p\left(1/\left(\frac{\hat{\phi }{\lambda }_j{m}_{jj}}{{\hat{\alpha }}_j^2}\right)\right)};\\ k_6& ={\hat{k}}_{GM}={\left(\prod _{j=1}^p\left(\frac{\hat{\phi }{\lambda }_j{m}_{jj}}{{\hat{\alpha }}_j^2}\right)\right)}^{1/p}.\end{array}$

5. The Monte Carlo simulation

In this section, we present a Monte Carlo simulation study to assess the performance of our proposed estimators under different evaluated situations, where MSE is considered as an assessment criterion.

5.1. The design of the simulation

The response variable of the GRM is generated using pseudo-random numbers from $G({\mu }_i,\phi )$ distribution with the log link function, where (23) $\begin{array}{r}{\hat{\mu }}_i=exp({\beta }_0+{\beta }_1{x}_{i1}+{\beta }_2{x}_{i2}+\dots +{\beta }_p{x}_{ip}),i=1,\dots ,n,j=1,\dots ,p.\end{array}$(23) The selection of parametric values in (23) is in such a way that $\sum _{j=1}^p{\beta }_j^2=1$, which is the common constraint in different simulation studies [7].

Following the work of McDonald and Galarneau [19], the explanatory variables are generated by $\begin{array}{r}{x}_{ij}=(1-{\rho }^2{)}^{1/2}{u}_{ij}+\rho u_{ij+1},i=1,2,\dots ,n,j=1,2,\dots ,p,\end{array}$Where $u_{ij}$ are pseudo-random numbers that are generated from the standard normal distribution and $\rho$ is specified so that the correlation between any two explanatory variables is given by ${\rho }^2$. To see the correlation effect in the simulation, we considered ${\rho }^2$ = 0.90, 0.95, 0.99, and 0.999. Additionally, we also consider the seven sets of sample sizes to examine the clear effect of multicollinearity, i.e. 50, 100, 150, 200, 300, 400 and 500. The values of $\beta$ affect the value of $F\beta =f$ which measures the correctness of the restrictions, and the performances of the estimators depend on the magnitude of $F\beta =f$. Therefore, following the work of Mansson et al. [11], the following restrictions are imposed on p = 4 and p = 8 which are respectively given as $\begin{array}{r}F=\left[\begin{array}{cc}1& 0\\ 1& -1\end{array}\begin{array}{cc}-2& 1\\ 1& -1\end{array}\right]\text{and}f=\left[\begin{array}{c}0\\ 0\end{array}\right]\end{array}$The restrictions for p = 8 are set to be $\begin{array}{r}F=\left[\begin{array}{cc}1& 0\\ 1& -1\end{array}\begin{array}{cc}-2& 1\\ 1& -1\end{array}\begin{array}{cc}-3& 1\\ -3& 1\end{array}\begin{array}{cc}1& 1\\ -2& 1\end{array}\right]\text{and}f=\left[\begin{array}{c}0\\ 0\end{array}\right]\end{array}$In the simulation procedure, the dispersion parameter is estimated to be the Pearson method, i.e. $\hat{\phi }=(n-p{)}^{-1}\sum _{i=1}^n\frac{{(y_i-{\hat{\mu }}_i)}^2}{{\hat{\mu }}_i^2}$ for each sample size and explanatory variables. For a combination of the various values, i.e. n, p and $\rho$ the generated data are repeated 2000 times, and the whole process is run 2000 times to compute the simulated MSE as follows [20,21]: $\begin{array}{r}MSE(\hat{\beta })=\frac{1}{R}\sum _{i=1}^R({\hat{\beta }}_{(i)}-\beta {)}^{\mathrm{\prime }}({\hat{\beta }}_{(i)}-\beta ),\end{array}$where R is the number of replications set to be 2000 times, and subscript (i) refers to the ith replication and ${\hat{\beta }}_{(i)}$ is the estimate of $\beta$ in the ith replication of the experiment.

5.2. Results and discussion

Table 1 and 2 represent the estimated MSEs of the GRM estimators. To assess the performance of the proposed estimators, we consider multiple factors, i.e. multicollinearity, sample size and different explanatory variables to monitor efficiency and compared the performance of the proposed estimator with some existing estimators. We can observe from tables that the new proposed estimators perform better than RMLE as well as MLE and GRRE. Specifically, the shrinkage parameter $k_3$ performed better in the sense of minimum MSE as compared to the other proposed estimators.

Table 1. Estimated MSE when p = 4.

				RGRREs
${\rho }^2$	MLE	RMLE	GRRE	$k_1$	$k_2$	$k_3$	$k_4$	$k_5$	$k_6$
n = 50
0.90	1.4789	2.1967	1.9924	1.9153	2.0181	1.8757	2.5679	2.5599	2.4921
0.95	3.8910	3.2059	3.7625	2.0216	2.4990	2.2085	2.5748	2.5718	2.5101
0.99	7.3485	4.3697	5.7827	2.1951	2.5403	2.4222	2.7294	2.6765	2.5468
0.999	43.5183	14.6022	29.2476	2.4615	4.5224	2.5017	3.9554	3.6973	2.7932
n = 100
0.90	1.3092	3.0432	2.3693	1.9758	2.0584	1.8733	2.1055	2.0388	1.9504
0.95	1.6662	3.2398	2.6326	2.0475	2.5240	1.8800	2.1072	2.0952	2.0878
0.99	6.8005	2.9696	5.4623	2.5423	2.5760	2.0794	3.0270	2.9427	2.6220
0.999	36.7372	14.7786	23.7815	2.6372	4.5797	2.7903	4.5325	3.7540	2.8669
n = 150
0.90	1.1884	2.8629	2.2395	1.9441	2.0100	1.8404	2.6578	2.6576	2.6397
0.95	1.4457	2.9898	2.4342	2.0846	2.5759	2.1295	2.7104	2.6935	2.4730
0.99	4.9081	2.6585	3.9219	2.6053	2.6092	2.4690	2.7541	2.7103	2.6916
0.999	23.3747	10.5039	16.2635	2.6515	4.2835	2.5312	4.4087	3.8625	2.7552
n = 200
0.90	1.1871	2.7906	2.1451	2.1121	2.3753	1.8313	2.7324	2.7276	2.5029
0.95	1.3821	2.7355	2.1730	2.2770	2.4454	2.0543	2.7862	2.7768	2.6372
0.99	2.7202	3.0946	2.9232	2.4845	2.5895	2.3382	2.9679	2.8286	2.6988
0.999	16.7432	8.4299	11.2302	2.6378	3.8387	2.5382	7.1910	6.1122	3.7215
n = 300
0.90	1.1087	2.6578	2.0452	2.0764	2.3183	1.8127	2.5060	2.3671	2.0926
0.95	1.2137	2.7104	2.1382	2.3120	2.5217	1.8164	2.8408	2.8066	2.6815
0.99	1.8563	2.8022	2.4557	2.5679	2.5491	2.4195	2.9336	2.8630	2.6928
0.999	9.6659	5.0502	7.0507	2.5841	2.6781	2.5274	8.8471	7.4460	4.1546
n = 400
0.90	1.0973	2.1227	1.7491	1.9456	1.9390	1.8032	2.6509	2.4280	2.1085
0.95	1.1491	3.1278	3.0260	2.0934	2.0866	1.8503	2.9479	2.8393	2.6177
0.99	1.7505	2.1841	2.0721	2.1074	2.4983	2.3133	3.0529	2.8738	2.6634
0.999	8.5366	5.4819	7.0731	2.9004	2.8624	2.4604	11.2707	8.8264	4.2570
n = 500
0.90	1.0606	2.5760	1.9238	2.1110	2.3553	1.7597	2.1733	2.1453	2.0702
0.95	1.1251	2.5714	1.9440	2.3796	2.4466	1.9030	3.0686	2.9228	2.6113
0.99	1.5790	2.7731	2.2372	2.4809	2.5047	1.9357	3.8728	3.4338	2.6500
0.999	6.3419	4.2120	5.0462	2.5305	2.5055	2.1980	11.2860	8.7452	3.9588

It can also be observed from Tables 1 and 2 that as we increase the levels of multicollinearity, there is a general increase in the estimated MSEs of the considered estimators. Moreover, findings also demonstrate that the unrestricted estimators, i.e. MLE and GRRE severely effect from the correlated explanatory variables. Table 2 shows that when severe multicollinearity exists among the explanatory variables, i.e. ${\rho }^2=0.999$ and n = 500, then the MSEs of the MLE, RMLE, GRRE and six proposed estimators are respectively given as 13.2751, 8.5290, 10.2307, 1.8389, 2.9298, 1.5251, 11.7882, 6.9484 and 2.5806. Among these, $k_3$ attains the minimum MSE as compared to others. So, we can say that our proposed $k_3$ are more resistant in the presence of severe multicollinearity as compared to other estimators. Moreover, explanatory variables also play a critical role in the performance of all the listed estimators. It is clearly observed from Tables 1 and 2 that as we increase the value of p from 4 to 8, the MSE of all the estimators may rapidly increase. For both the value of p, we observed that our proposed restricted ridge parameters outperform the RMLE and other estimation methods in all of the evaluated situations. Thus we can say that the proposed RGRRE are significantly decreasing the MSE. The sample size is also an important factor in judging the performance of any estimator. Findings clarify that as we increase the sample size, the estimated MSE gradually decreases for all the estimators under study, which is considered as an important property of any estimator. Usually, for the estimation of unknown parameters, the most robust option is $k_3$ as compared to other estimation methods in the presence of severe multicollinearity.

Table 2. Estimated MSE when p = 8.

				RGRREs
${\rho }^2$	MLE	RMLE	GRRE	$k_1$	$k_2$	$k_3$	$k_4$	$k_5$	$k_6$
n = 50
0.90	2.8146	2.2832	2.7228	1.8714	2.0602	1.4898	2.5839	2.3330	1.9981
0.95	4.0495	2.6736	3.3029	2.1198	2.2355	1.8010	2.6271	2.4852	2.1312
0.99	18.6739	10.6778	14.0816	2.5096	2.4149	2.3679	2.6337	2.5540	2.4010
0.999	169.2791	76.4868	117.5995	2.5981	2.5079	2.5289	6.2984	4.1685	2.5127
n = 100
0.90	2.2259	1.8457	1.9329	1.6841	1.6205	1.4915	1.7722	1.6727	1.5966
0.95	4.4072	3.7213	3.7621	1.9697	1.9027	1.5691	2.7471	2.5046	1.9032
0.99	10.8583	6.4468	8.4847	2.0701	2.2973	1.6420	3.1911	2.5662	2.2418
0.999	96.1702	55.4801	69.8863	2.4005	3.8174	2.0958	14.9974	9.4114	3.3787
n = 150
0.90	1.7348	1.6156	1.9609	1.5398	1.5085	1.3494	2.0746	1.8246	1.5094
0.95	3.2939	1.8975	2.5863	1.5791	1.5438	1.3770	2.3257	1.9150	1.5161
0.99	5.4425	3.9536	4.5621	1.9168	3.2907	1.3830	8.6014	6.3262	3.1231
0.999	53.7690	31.2228	38.6210	2.4240	11.6963	1.6647	57.9513	36.6693	8.7993
n = 200
0.90	1.6524	1.8763	2.1553	1.4967	1.2904	1.2787	1.5247	1.4071	1.3049
0.95	2.9910	2.9477	3.5075	1.6546	1.5966	1.3720	1.7594	1.5906	1.4320
0.99	4.9809	3.7434	4.9794	1.7178	1.6017	1.4051	3.1686	2.3809	1.6411
0.999	37.5602	20.2644	27.1838	1.8736	2.4681	1.5266	22.8912	13.7120	4.4784
n = 300
0.90	1.5774	1.5470	1.2199	1.3743	1.3107	1.3395	1.6131	1.3995	1.2945
0.95	2.9773	1.8867	2.4356	1.4982	1.4506	1.4590	1.8893	1.7061	1.5895
0.99	4.5795	3.7375	4.3151	1.6286	1.6612	1.4761	2.2438	1.9016	1.5903
0.999	33.3521	17.0757	22.3628	2.2572	5.4412	1.5676	8.1255	5.1348	2.3265
n = 400
0.90	1.4718	1.8322	1.0945	1.2385	1.2007	1.1579	1.5163	1.3005	1.1801
0.95	2.5759	2.0734	1.8058	1.7869	1.9333	1.4379	3.1571	2.5993	1.9395
0.99	3.4411	2.5509	3.2413	2.1625	2.1227	1.4833	5.0287	3.5141	2.0598
0.999	16.1596	11.1361	11.8065	2.2135	7.9930	1.7991	38.6583	22.8370	6.2168
n = 500
0.90	1.4010	2.6842	2.7475	1.1973	1.0879	1.0986	1.3530	1.1741	1.0878
0.95	2.8418	2.6574	2.8247	1.6185	1.5657	1.3919	1.7727	1.6507	1.5519
0.99	3.8747	2.8997	3.4940	1.7405	1.8112	1.5162	3.0535	2.4248	1.7771
0.999	13.2751	8.5290	10.2307	1.8389	2.9298	1.5251	11.7882	6.9484	2.5806

6. An empirical application: hydrocarbon escape data

In this section, we evaluate the performance of the proposed estimator using a real application. For this purpose, we consider a hydrocarbon dataset, which is taken from Weisberg [22]. When petrol is pumped into tanks, hydrocarbons escape into the atmosphere. For the reduction of pollution in the atmosphere, different devices are installed for the absorption of vapours. To evaluate their effectiveness, 32 laboratory experiments were conducted without using the devices. There were four explanatory variables that are involved in this laboratory experiment and the description of the given variables is as follows: The quantity y (in grams) of hydrocarbon escaping was measured as a function of the tank temperature $x_1$ (in ${}^{o}F$), the temperature $x_2$ (in ${}^{o}F$) of the petrol pumped in, the initial pressure $x_3$ in the tank and the pressure $x_4$ of the petrol pumped in (both in pounds per square inch). Before further proceeding, it is very crucial to find the probability distribution of the response variable y to identify the appropriate regression model. For this purpose, we use three important tests, namely, Anderson–Darling, Cramer–Von Mises and Pearson ${\chi }^2$ test (for detailed description, we recommend the following studies: [23,24]). The results of Table 3 showed that the hydrogen escape dataset is well fitted to the gamma distribution due to its maximum p-value as compared to other competitive distributions. More specifically, it can be seen that the test statistic (p-value) of Cramer–Von Mises found to be 0.0459 (0.5811) clearly signifying that the considered dataset is well fitted to the gamma distribution. The bivariate correlation among the four explanatory variables is displayed in Table 4. One can be clearly seen from Table 4 that there is a high correlation among all of the four explanatory variables, which clearly indicates that the dataset is highly multicollinear.

Table 3. Distribution of goodness of fit test for hydrogen escape data.

		Probability distributions
Goodness of fit tests		Normal	Exponential	Gamma	Weibull
AD	Statistic	0.5334	7.5994	0.3000	0.5937
	p-value	0.1725	0.0000	0.6152	0.1235
CVM	Statistic	0.0727	1.5704	0.0459	0.0836
	p-value	0.2605	0.0000	0.5811	0.1776
Pearson ${\chi }^2$	Statistic	9.5000	48.500	3.0000	12.000
	p-value	0.0907	0.0000	0.6999	0.0347

Note: AD = Anderson Darling; CVM = Crammer Von Mises.

Table 4. Correlation matrix.

	$x_1$	$x_2$	$x_3$	$x_4$
$x_1$	1
$x_2$	0.7742	1
$x_3$	0.9554	0.7815	1
$x_4$	0.9337	0.8374	0.985	1

Based on the analysis of the dataset, the estimated coefficients and relative efficiencies are summarized in Table 5 to judge the performance of the MLE, RMLE and RGRREs. The restriction matrix for the considered dataset is to be $F=[0,1,1,1,1]$ with $f=[0]$. Since relative efficiency is used in the example for assessment purpose, particularly defined as $e({\hat{\beta }}_{MLE},{\hat{\beta }}_i)=\frac{SMSE({\hat{\beta }}_i)}{SMSE({\hat{\beta }}_{MLE})}$ where ${\hat{\beta }}_i({\hat{\beta }}_{RMLE},\hat{\beta }(k),{\hat{\beta }}_F(k_1)-{\hat{\beta }}_F(k_6))$ is any biased estimator. Thus $e({\hat{\beta }}_{MLE},{\hat{\beta }}_i)<1$ indicate improved precision relative to MLE, and $e({\hat{\beta }}_{MLE},{\hat{\beta }}_i)>1$ indicate worse performance. From Table 5, the RMLE (${\hat{\beta }}_{RMLE}$), GRRE ($\hat{\beta }(k)$) and proposed RGRREs (${\hat{\beta }}_F(k_1)-{\hat{\beta }}_F(k_6)$) have smaller SMSE than MLE (${\hat{\beta }}_{MLE}$). Moreover, the minimum value of $e({\hat{\beta }}_{MLE},{\hat{\beta }}_i)$ suggests the best estimator, among others. The application result shows that the performance of the proposed RGRRE outperforms the RMLE and GRRE in the presence of high, but imperfect multicollinearity. In Table 5, the results from the GRM are shown in terms of the estimated parameter and relative efficiency for different estimators. It can be noted that the MSE is most extensive for ${\hat{\beta }}_{MLE}$ and $\hat{\beta }(k)$ indicating the poor performance in case of severe multicollinearity and many explanatory variables as shown in the simulation study. The second worst estimator is the ${\hat{\beta }}_{RMLE}$. The estimator that minimizes the MSE is the ${\hat{\beta }}_F(k_6)$.

Table 5. Estimated parameter and relative efficiency for different estimators.

	RGRREs
	${\hat{\beta }}_{MLE}$	${\hat{\beta }}_{RMLE}$	$\hat{\beta }(k)$	${\hat{\beta }}_F(k_1)$	${\hat{\beta }}_F(k_2)$	${\hat{\beta }}_F(k_3)$	${\hat{\beta }}_F(k_4)$	${\hat{\beta }}_F(k_5)$	${\hat{\beta }}_F(k_6)$
(Intercept)	0.0342	0.0306	0.0342	0.0305	0.0304	0.0305	0.0305	0.0305	0.0302
$x_1$	−0.0026	0.0062	−0.0025	0.0064	0.0071	0.0066	0.0065	0.0068	0.0077
$x_2$	−0.0025	0.0015	−0.0026	0.0014	0.0008	0.0012	0.0013	0.0011	0.0001
$x_3$	0.0101	0.0171	0.0095	0.0162	0.0121	0.0152	0.0155	0.0141	0.0071
$x_4$	−0.0126	−0.0248	−0.012	−0.024	−0.0201	−0.0231	−0.0233	−0.0219	−0.015
$e({\hat{\beta }}_{MLE},{\hat{\beta }}_i)$	1.0000	0.9726	0.9080	0.8941	0.5930	0.8177	0.8365	0.7281	0.3067

7. Conclusion

In this paper, we introduced a new RGRRE for the estimation of unknown parameters of the GRM in the presence of mild to severe multicollinearity. We also proposed some methods to choose the ridge parameter k for the RGRRE. A Monte Carlo simulation study has been designed to evaluate the performance of the proposed RGRREs and compared it with other estimators under different evaluated conditions where MSE is considered as an assessment criterion. Moreover, to illustrate the benefits of using the proposed estimator, we also consider an empirical application. From the simulation and real application results, we conclude that the performance of RGRRE under different ridge parameters is better than the MLE, RMLE and GRRE. Specifically, the RGRRE with parameter $k_3$ is the most robust and recommended estimator in the sense of minimum MSE. Since Monte Carlo simulation results proved $k_3$ as the most resistant estimator in the presence of severe multicollinearity. Hence, we suggest the researchers to use RGRRE with ridge parameter $k_3$ and $k_6$ for the estimation of GRM in the presence of severe multicollinearity.

Disclosure statement

No potential conflict of interest was reported by the author(s).