Performance of Existing Biased Estimators and the Respective Predictors in a Misspecified Linear Regression Model

Manickavasagar Kayanan^1,2, Pushpakanthie Wijekoon³

¹Deparment of Physical Science, Vavuniya Campus of the University of Jaffna, Vavuniya, Sri Lanka.
²Postgraduate Institute of Science, University of Peradeniya, Peradeniya, Sri Lanka.
³Department of Statistics and Computer Science, University of Peradeniya, Peradeniya, Sri Lanka.

DOI: 10.4236/ojs.2017.75062                

Abstract

In this paper, the performance of existing biased estimators (Ridge Estimator (RE), Almost Unbiased Ridge Estimator (AURE), Liu Estimator (LE), Almost Unbiased Liu Estimator (AULE), Principal Component Regression Estimator (PCRE), r-k class estimator and r-d class estimator) and the respective predictors were considered in a misspecified linear regression model when there exists multicollinearity among explanatory variables. A generalized form was used to compare these estimators and predictors in the mean square error sense. Further, theoretical findings were established using mean square error matrix and scalar mean square error. Finally, a numerical example and a Monte Carlo simulation study were done to illustrate the theoretical findings. The simulation study revealed that LE and RE outperform the other estimators when weak multicollinearity exists, and RE, r-k class and r-d class estimators outperform the other estimators when moderated and high multicollinearity exist for certain values of shrinkage parameters, respectively. The predictors based on the LE and RE are always superior to the other predictors for certain values of shrinkage parameters.

Keywords

Misspecified Regression Model, Generalized Biased Estimator, Generalized Predictor, Mean Square Error Matrix, Scalar Mean Square Error

1. Introduction

It is well known that the misspecification of the linear model is unavoidable in practical situations. Misspecification may occur due to including some irrelevant explanatory variables or excluding some relevant explanatory variables. Excluding some relevant explanatory variables from a regression model causes these variables to become part of the error term. In this case the mean of error term of the model is not zero. Furthermore, the excluded variables may be correlated with the variables in the model. According to the assumptions of linear regression model, the error term of the model should be independently and identically normally distributed with mean zero and variance ${\sigma }^2$ . Therefore, one or more assumptions of the linear regression model will be violated when the model is misspecified, and hence the estimators become biased and inconsistent.

Further, it is well known that the ordinary least square estimator (OLSE) does not hold its desirable properties if multicollinearity exists among the explanatory variables in the regression model. To overcome this problem, biased estimators based on the sample model $y=X\beta +\epsilon$ , or by combining sample model with the exact or stochastic restrictions have been used in the literature. The motivation of this article is to examine the performance of the existing biased estimators in the misspecified linear regression model when multicollinearity exists.

Sarkar [1] examined the consequences of omission of some relevant explanatory variables from a linear regression model when multicollinearity exists among the explanatory variables. Recently, Şiray [2] and Wu [3] discussed the efficiency of r-d class estimator and r-k class estimator over some existing estimators, respectively. Teräsvirta [4] was discussed the case of biased estimation with stochastic linear restrictions in the misspecified regression model due to including an irrelevant variable with incorrectly specified prior information. Later, the efficiency of Mixed Regression Estimator (MRE) under misspecified regression model due to excluding relevant variable with correctly specified prior information was discussed by Mittelhmmer [5] , Ohtani and Honda [6] , Kadiyala [7] and Trenkler and Wijekoon [8] . Further, the superiority of MRE over the Ordinary Least Squares Estimator (OLSE) under the misspecified regression model with incorrectly specified sample and prior information was discussed by Wijekoon and Trenkler [9] . Hubert and Wijekoon [10] have considered the improvement of Liu estimator under a misspecified regression model with stochastic restrictions.

In this paper, the performance of existing biased estimators of the linear regression model based on the sample information such as Principal Component Regression Estimator (PCRE) introduced by Massy [11] , Ridge Estimator (RE) defined by Hoerl and Kennard [12] , r-k class estimator proposed by Baye and Parker [13] , Almost Unbiased Ridge Estimator (AURE) proposed by Singh et al. [14] , Liu Estimator (LE) proposed by Liu [15] , Almost Unbiased Liu Estimator (AULE) proposed by Akdeniz and Kaçiranla r [16] , r-d class estimator proposed by Kaçıranlar and Sakallıoğlu [17] were examined under misspecified regression model without combining any prior information to the sample model. A generalized form to represent all the above estimators was used for comparing these estimators and their respective predictors easily.

The rest of this article is organized as follows. The model specification and respective OLSE are written in section 2. In section 3, generalized form to represent the estimators under the misspecified regression model is proposed. In section 4, the Mean Square Error Matrix (MSEM) and Scalar Mean Square Error (SMSE) comparison between two generalized estimators and their respective predictors are considered. In section 5, the numerical example and Monte Carlo simulation are given to illustrate the theoretical results in SMSE criterion. Finally, some concluding remarks are stated in section 6. The references and Appendix are given at the end of the paper.

2. Model Specification

Assume that the true regression model is given by

$y=X_1{\beta }_1+X_2{\beta }_2+\epsilon =X_1{\beta }_1+\delta +\epsilon$ (2.1)

where $y$ is the $n\times 1$ vector of observations on the dependent variable, $X_1$ and $X_2$ are the $n\times l$ and $n\times p$ matrices of observations on the $m=l+p$ regressors, ${\beta }_1$ and ${\beta }_2$ are the $l\times 1$ and $p\times 1$ vectors of unknown coefficients, $\epsilon$ is the $n\times 1$ vector of disturbances with mean vector zero $\left(E\left(\epsilon \right)=0\right)$ and dispersion matrix ${\sigma }^{2}I$ $\left(E\left(\epsilon {\epsilon }^{\prime }\right)=\Omega ={\sigma }^{2}I\right)$ .

Let us say that the researcher misspecifies the regression model by excluding $p$ regressors as

$y=X_1{\beta }_1+u$ (2.2)

According to Singh et al. [14] , by applying the spectral decomposition of the symmetric matrix ${X^{\prime }}_1{X}_1$ (since ${X^{\prime }}_1{X}_1$ is a positive definite matrix) we have $T^{\prime }{X^{\prime }}_1{X}_{1}T=\Lambda =\text{diag}\left({\lambda }_1,\cdots ,{\lambda }_l\right)$ , where $T=\left(t_1,t_2,\cdots ,t_l\right)$ is the orthogonal matrix and ${\lambda }_i>0$ being the i^th eigenvalue of ${X^{\prime }}_1{X}_1$ . Let $T_r=\left(t_1,t_2,\cdots ,t_r\right)$ be the remaining column of $T$ having deleted $l-r$ columns where $r\le l$ . Hence, ${T^{\prime }}_r{X^{\prime }}_1{X}_1{T}_r={\Lambda }_r=\text{diag}\left({\lambda }_1,\cdots ,{\lambda }_r\right)$ .

Let $Z=X_{1}T$ and $\gamma =T^{\prime }{\beta }_1$ then models (2.1) and (2.2) can be written in canonical form as

$y=Z\gamma +\delta +\epsilon$ (2.3)

$y=Z\gamma +u$ (2.4)

The OLS estimator of model (2.4) is given by

$\hat{\gamma }={\left(Z^{\prime }Z\right)}^{-1}{Z}^{\prime }y={\Lambda }^{-1}{Z}^{\prime }y$ (2.5)

Using (2.3), $\hat{\gamma }$ can be written as

$\hat{\gamma }={\Lambda }^{-1}{Z}^{\prime }\left(Z\gamma +\delta +\epsilon \right)=\gamma +{\Lambda }^{-1}{Z}^{\prime }\left(\delta +\epsilon \right)$ (2.6)

Hence, the expectation vector and the dispersion matrix of $\hat{\gamma }$ are given by

$E\left(\hat{\gamma }\right)=\gamma +{\Lambda }^{-1}{Z}^{\prime }\delta$ (2.7)

and

$D\left(\hat{\gamma }\right)={\sigma }^2{\left(Z^{\prime }Z\right)}^{-1}={\sigma }^2{\Lambda }^{-1}$ (2.8)

respectively.

3. Modified Biased Estimators, Predictors and Its Generalized Form

To combat multicollinearity several researchers introduce different types of biased estimators in place of OLSE, and seven such estimators are RE, AURE, LE, ALUE, PCRE, r-k class estimator and r-d class estimator given bellow respectively:

${\hat{\beta }}_k={\left(X^{\prime }X+kI\right)}^{-1}{X}^{\prime }X\hat{\beta }$ (3.1)

${\hat{\beta }}_{AURE}=\left(I-k^2{\left(X^{\prime }X+kI\right)}^{-2}\right)\hat{\beta }$ (3.2)

${\hat{\beta }}_d={\left(X^{\prime }X+I\right)}^{-1}\left(X^{\prime }X+dI\right)\hat{\beta }$ (3.3)

${\hat{\beta }}_{AULE}=\left(I-{\left(1-d\right)}^2{\left(X^{\prime }X+I\right)}^{-2}\right)\hat{\beta }$ (3.4)

${\hat{\beta }}_{PCR}=T_r{\left({T^{\prime }}_r{X}^{\prime }X{T}_r\right)}^{-1}{T^{\prime }}_r{X}^{\prime }y$ (3.5)

${\hat{\beta }}_{rk}=T_r{\left({T^{\prime }}_r{X}^{\prime }X{T}_r+k{I}_r\right)}^{-1}{T^{\prime }}_r{X}^{\prime }y$ (3.6)

${\hat{\beta }}_{rd}=T_r{\left({T^{\prime }}_r{X}^{\prime }X{T}_r+I_r\right)}^{-1}\left(I_r+d{\left({T^{\prime }}_r{X}^{\prime }X{T}_r\right)}^{-1}\right){T^{\prime }}_r{X}^{\prime }y$ (3.7)

where $X=\left[X_1{X}_2\right]$ , $k>0$ , $0<d<1$ and $\hat{\beta }$ is the OLS estimator of the model (2.1).

Further, Xu and Yang [18] showed that Equations (3.5)-(3.7) could be rewritten as

${\hat{\beta }}_{PCR}=T_r{T^{\prime }}_r\hat{\beta }$ (3.8)

${\hat{\beta }}_{rk}=T_r{T^{\prime }}_r{\left(X^{\prime }X+kI\right)}^{-1}{X}^{\prime }X\hat{\beta }$ (3.9)

${\hat{\beta }}_{rd}=T_r{T^{\prime }}_r{\left(X^{\prime }X+I\right)}^{-1}\left(X^{\prime }X+dI\right)\hat{\beta }$ (3.10)

In the case of misspecification, the RE, AURE, LE, AULE, PCRE, r-k class estimator and r-d class estimator for the model (2.4) can be written as

${\hat{\gamma }}_k={\left(Z^{\prime }Z+kI\right)}^{-1}{Z}^{\prime }Z\hat{\gamma }={\left(\Lambda +kI\right)}^{-1}\Lambda \hat{\gamma }=A_k\hat{\gamma }$ (3.11)

${\hat{\gamma }}_{AURE}=\left(I-k^2{\left(Z^{\prime }Z+kI\right)}^{-2}\right)\hat{\gamma }=\left(I-k^2{\left(\Lambda +kI\right)}^{-2}\right)\hat{\gamma }=A_k^{\text{*}}\hat{\gamma }$ (3.12)

${\hat{\gamma }}_d={\left(Z^{\prime }Z+I\right)}^{-1}\left(Z^{\prime }Z+dI\right)\hat{\gamma }={\left(\Lambda +I\right)}^{-1}\left(\Lambda +dI\right)\hat{\gamma }=F_d\hat{\gamma }$ (3.13)

${\hat{\gamma }}_{AULE}=\left(I-{\left(1-d\right)}^2{\left(Z{Z}^{\prime }+I\right)}^{-2}\right)\hat{\gamma }=\left(I-{\left(1-d\right)}^2{\left(\Lambda +I\right)}^{-2}\right)\hat{\gamma }=F_d^{\text{*}}\hat{\gamma }$ (3.14)

${\hat{\gamma }}_{PCR}=T_r{T^{\prime }}_r\hat{\gamma }=W_r\hat{\gamma }$ (3.15)

${\hat{\gamma }}_{rk}=T_r{T^{\prime }}_r{\left(\Lambda +kI\right)}^{-1}\Lambda \hat{\gamma }=W_{rk}\hat{\gamma }$ (3.16)

${\hat{\gamma }}_{rd}=T_r{T^{\prime }}_r{\left(\Lambda +I\right)}^{-1}\left(\Lambda +dI\right)\hat{\gamma }=W_{rd}\hat{\gamma }$ (3.17)

respectively.

where $A_k={\left(\Lambda +kI\right)}^{-1}\Lambda$ , $A_k^{\text{*}}=\left(I-k^2{\left(\Lambda +kI\right)}^{-2}\right)$ , $F_d={\left(\Lambda +I\right)}^{-1}\left(\Lambda +dI\right)$ , $F_d^{\text{*}}=\left(I-{\left(1-d\right)}^2{\left(\Lambda +I\right)}^{-2}\right)$ , $W_r=T_r{T^{\prime }}_r$ , $W_{rk}=T_r{T^{\prime }}_r{\left(\Lambda +kI\right)}^{-1}\Lambda$ and $W_{rd}=T_r{T^{\prime }}_r{\left(\Lambda +I\right)}^{-1}\left(\Lambda +dI\right)$ .

It is clear that $A_k$ and $F_d$ are positive definite, and $W_r$ , $W_{rk}$ and $W_{rd}$ are non-negative definite.

Now consider $\begin{array}{c}{A}_k^{\text{*}}=\left(I-k^2{\left(\Lambda +kI\right)}^{-2}\right)={\left(\Lambda +kI\right)}^{-2}\left({\left(\Lambda +kI\right)}^2-k^2\right)\\ ={\left(\Lambda +kI\right)}^{-2}\Lambda \left(\Lambda +2kI\right)>0\end{array}$

and $\begin{array}{c}{F}_d^{\text{*}}=\left(I-{\left(1-d\right)}^2{\left(\Lambda +I\right)}^{-2}\right)={\left(\Lambda +I\right)}^{-2}\left({\left(\Lambda +I\right)}^2-{\left(1-d\right)}^{2}I\right)\\ ={\left(\Lambda +I\right)}^{-2}\left(\Lambda +dI\right)\left(\Lambda +\left(2-d\right)I\right)>0;0<d<1\end{array}$

Hence, $A_k^{\text{*}}$ and $F_d^{\text{*}}$ are also positive definite.

Since RE, AURE, LE, AULE, PCRE, r-k class estimator and r-d class estimator are based on $\hat{\gamma }$ so we can use the following generalized form:

${\hat{\gamma }}_{\left(j\right)}=G_{\left(j\right)}\hat{\gamma }$ (3.18)

where $G_{\left(j\right)}$ is positive definite matrix if it stands for $A_k$ , $A_k^{\text{*}}$ , $F_d$ and $F_d^{\text{*}}$ , and it is non-negative definite matrix if it stands for $W_r,W_{rk}$ and $W_{rd}$ .

The expectation vector, bias vector, dispersion matrix and the mean square error matrix can be calculated with

$E\left({\hat{\gamma }}_{\left(j\right)}\right)=G_{\left(j\right)}E\left(\hat{\gamma }\right)=G_{\left(j\right)}\left(\gamma +{\Lambda }^{-1}{Z}^{\prime }\delta \right)$

$\begin{array}{c}Bias\left({\hat{\gamma }}_{\left(j\right)}\right)=E\left({\hat{\gamma }}_{\left(j\right)}-\gamma \right)=G_{\left(j\right)}\left(\gamma +{\Lambda }^{-1}{Z}^{\prime }\delta \right)-\gamma \\ =\left(G_{\left(j\right)}-I\right)\gamma +G_{\left(j\right)}{\Lambda }^{-1}{Z}^{\prime }\delta \end{array}$ (3.19)

$D\left({\hat{\gamma }}_{\left(j\right)}\right)=G_{\left(j\right)}D\left(\hat{\gamma }\right){G^{\prime }}_{\left(j\right)}={\sigma }^2{G}_{\left(j\right)}{\Lambda }^{-1}{G^{\prime }}_{\left(j\right)}$ (3.20)

$\begin{array}{l}MSEM\left({\hat{\gamma }}_{\left(j\right)}\right)=E\left({\hat{\gamma }}_{\left(j\right)}-\gamma \right){\left({\hat{\gamma }}_{\left(j\right)}-\gamma \right)}^{\prime }=D\left({\hat{\gamma }}_{\left(j\right)}\right)+Bias\left({\hat{\gamma }}_{\left(j\right)}\right)Bias{\left({\hat{\gamma }}_{\left(j\right)}\right)}^{\prime }\\ ={\sigma }^2{G}_{\left(j\right)}{\Lambda }^{-1}{G^{\prime }}_{\left(j\right)}+\left(\left(G_{\left(j\right)}-I\right)\gamma +G_{\left(j\right)}{\Lambda }^{-1}{Z}^{\prime }\delta \right){\left(\left(G_{\left(j\right)}-I\right)\gamma +G_{\left(j\right)}{\Lambda }^{-1}{Z}^{\prime }\delta \right)}^{\prime }\end{array}$ (3.21) (3)

Based on 3.19 to 3.21, the respective expectation vector, bias vector and dispersion matrix of the RE, AURE, LE, AULE, PCR, r-k class estimator and r-d class estimator can easily be obtained and given in Table A1 in the Appendix.

By using the approach of Kadiyala [7] , and Equations ((2.3) and (2.4)), the generalized prediction function can be defined as follows:

$y_0=Z\gamma +\delta$ (3.22)

${\hat{y}}_{\left(j\right)}=Z{\hat{\gamma }}_{\left(j\right)}$ (3.23)

where $y_0$ is the actual value and ${\hat{y}}_{\left(j\right)}$ is the corresponding predictor value.

The MSEM of the generalized predictor is given by

$\begin{array}{l}MSEM\left({\hat{y}}_{\left(j\right)}\right)=E\left({\hat{y}}_{\left(j\right)}-y_0\right){\left({\hat{y}}_{\left(j\right)}-y_0\right)}^{\prime }\\ =Z\left(MSEM\left({\hat{\gamma }}_{\left(j\right)}\right)\right)Z^{\prime }-Z\left(Bias\left({\hat{\gamma }}_{\left(j\right)}\right)\right){\delta }^{\prime }-\delta {\left(Bias\left({\hat{\gamma }}_{\left(j\right)}\right)\right)}^{\prime }{Z}^{\prime }+\delta {\delta }^{\prime }\end{array}$ (3.24)

Note that the predictors based on the OLSE, RE, AURE, LE, AULE, PCRE, r-k class estimator and r-d class estimator are denoted by $\hat{y},{\hat{y}}_k,{\hat{y}}_{AURE},{\hat{y}}_d,{\hat{y}}_{AULE},{\hat{y}}_{PCR},{\hat{y}}_{rk}$ and ${\hat{y}}_{rd}$ respectively.

4. Mean Square Error Comparisons

4.1. Mean Square Error Matrix (MSEM) Comparison of Generalized Estimators

If two generalized biased estimators ${\hat{\gamma }}_{\left(i\right)}$ and ${\hat{\gamma }}_{\left(j\right)}$ are given, the estimator ${\hat{\gamma }}_{\left(j\right)}$ is said to be superior to ${\hat{\gamma }}_{\left(i\right)}$ with respect to MSEM sense if and only if $MSEM\left({\hat{\gamma }}_{\left(i\right)}\right)-MSEM\left({\hat{\gamma }}_{\left(j\right)}\right)\ge 0$ .

Let us consider

$\begin{array}{l}MSEM\left({\hat{\gamma }}_{\left(i\right)}\right)-MSEM\left({\hat{\gamma }}_{\left(j\right)}\right)\\ =\left(\left(G_{\left(i\right)}-I\right)\gamma +G_{\left(i\right)}{\Lambda }^{-1}{Z}^{\prime }\delta \right){\left(\left(G_{\left(i\right)}-I\right)\gamma +G_{\left(i\right)}{\Lambda }^{-1}{Z}^{\prime }\delta \right)}^{\prime }\\ -\left(\left(G_{\left(j\right)}-I\right)\gamma +G_{\left(j\right)}{\Lambda }^{-1}{Z}^{\prime }\delta \right){\left(\left(G_{\left(j\right)}-I\right)\gamma +G_{\left(j\right)}{\Lambda }^{-1}{Z}^{\prime }\delta \right)}^{\prime }\\ +{\sigma }^2{G}_{\left(i\right)}{\Lambda }^{-1}{G^{\prime }}_{\left(i\right)}-{\sigma }^2{G}_{\left(j\right)}{\Lambda }^{-1}{G^{\prime }}_{(j)}\end{array}$

Now let $D_{\left(i,j\right)}={\sigma }^2\left(G_{\left(i\right)}{\Lambda }^{-1}{G^{\prime }}_{\left(i\right)}-G_{\left(j\right)}{\Lambda }^{-1}{G^{\prime }}_{\left(j\right)}\right)$ , $b_{\left(i\right)}=\left(\left(G_{\left(i\right)}-I\right)\gamma +G_{\left(i\right)}{\Lambda }^{-1}{Z}^{\prime }\delta \right)$ , $b_{\left(j\right)}=\left(\left(G_{\left(j\right)}-I\right)\gamma +G_{\left(j\right)}{\Lambda }^{-1}{Z}^{\prime }\delta \right)$ , then the above difference can be written as

${\text{Δ}}_{\left(i,j\right)}=MSEM\left({\hat{\gamma }}_{\left(i\right)}\right)-MSEM\left({\hat{\gamma }}_{\left(j\right)}\right)=D_{\left(i,j\right)}+b_{\left(i\right)}{b^{\prime }}_{\left(i\right)}-b_{\left(j\right)}{b^{\prime }}_{\left(j\right)}$ (4.1)

The following theorem can be stated for superiority of ${\hat{\gamma }}_{\left(j\right)}$ over ${\hat{\gamma }}_{\left(i\right)}$ with respect to the MSEM criterion.

Theorem 1: If $G_{\left(i\right)}$ is positive definite, ${\hat{\gamma }}_{\left(j\right)}$ is superior to ${\hat{\gamma }}_{\left(i\right)}$ in MSEM sense when the regression model is misspecified due to excluding relevant variables if and only if ${\lambda }_{\text{*}}<1$ and ${b^{\prime }}_{\left(j\right)}{\left(D_{\left(i,j\right)}+b_{\left(i\right)}{b^{\prime }}_{\left(i\right)}\right)}^{-1}{b}_{\left(j\right)}\le 1$ , where ${\lambda }_{\text{*}}$ is the largest eigenvalue of $\left(G_{\left(j\right)}{\Lambda }^{-1}{G^{\prime }}_{\left(j\right)}\right){\left(G_{\left(i\right)}{\Lambda }^{-1}{G^{\prime }}_{\left(i\right)}\right)}^{-1}$ , $D_{\left(i,j\right)}={\sigma }^2\left(G_{\left(i\right)}{\Lambda }^{-1}{G^{\prime }}_{\left(i\right)}-G_{\left(j\right)}{\Lambda }^{-1}{G^{\prime }}_{\left(j\right)}\right)$ , $b_{\left(i\right)}=\left(\left(G_{\left(i\right)}-I\right)\gamma +G_{\left(i\right)}{\Lambda }^{-1}{Z}^{\prime }\delta \right)$ and $b_{\left(j\right)}=\left(\left(G_{\left(j\right)}-I\right)\gamma +G_{\left(j\right)}{\Lambda }^{-1}{Z}^{\prime }\delta \right)$ .

Proof: Assume that $G_{\left(i\right)}$ is positive definite, which implies $G_{\left(i\right)}{\Lambda }^{-1}{G^{\prime }}_{\left(i\right)}>0$ .

Due to Lemma 3 (see Appendix), $G_{\left(i\right)}{\Lambda }^{-1}{G^{\prime }}_{\left(i\right)}>G_{\left(j\right)}{\Lambda }^{-1}{G^{\prime }}_{\left(j\right)}$ if ${\lambda }_{\text{*}}<1$ , where ${\lambda }_{\text{*}}$ is the largest eigenvalue of $\left(G_{\left(j\right)}{\Lambda }^{-1}{G^{\prime }}_{\left(j\right)}\right){\left(G_{\left(i\right)}{\Lambda }^{-1}{G^{\prime }}_{\left(i\right)}\right)}^{-1}$ .

Hence, according to Lemma 2 (see Appendix), ${\text{Δ}}_{\left(i,j\right)}$ is non-negative if and only if ${b^{\prime }}_{\left(j\right)}{\left(D_{\left(i,j\right)}+b_{\left(i\right)}{b^{\prime }}_{\left(i\right)}\right)}^{-1}{b}_{\left(j\right)}\le 1$ , which completes the proof.

4.2. Scalar Mean Square Error (SMSE) Comparison of Generalized Estimators

If two generalized biased estimators ${\hat{\gamma }}_{\left(i\right)}$ and ${\hat{\gamma }}_{\left(j\right)}$ are given, the estimator ${\hat{\gamma }}_{\left(j\right)}$ is said to be superior to ${\hat{\gamma }}_{\left(i\right)}$ with respect to SMSE sense if and only if $SMSE\left({\hat{\gamma }}_{\left(i\right)}\right)-SMSE\left({\hat{\gamma }}_{\left(j\right)}\right)>0$ .

The following theorem can be stated for superiority of ${\hat{\gamma }}_{\left(j\right)}$ over ${\hat{\gamma }}_{\left(i\right)}$ with respect to the SMSE criterion.

Theorem 2: ${\hat{\gamma }}_{\left(j\right)}$ is superior to ${\hat{\gamma }}_{\left(i\right)}$ when the regression model is misspecified due to excluding relevant variable with respect to SMSE sense if and only if

$\frac{tr\left(b_{\left(j\right)}{b^{\prime }}_{\left(j\right)}-b_{\left(i\right)}{b^{\prime }}_{\left(i\right)}\right)}{tr\left(D_{\left(i,j\right)}\right)}<1$

where $D_{\left(i,j\right)}={\sigma }^2\left(G_{\left(i\right)}{\Lambda }^{-1}{G^{\prime }}_{\left(i\right)}-G_{\left(j\right)}{\Lambda }^{-1}{G^{\prime }}_{\left(j\right)}\right)$ , $b_{\left(i\right)}=\left(\left(G_{\left(i\right)}-I\right)\gamma +G_{\left(i\right)}{\Lambda }^{-1}{Z}^{\prime }\delta \right)$ and

$b_{\left(j\right)}=\left(\left(G_{\left(j\right)}-I\right)\gamma +G_{\left(j\right)}{\Lambda }^{-1}{Z}^{\prime }\delta \right)$ .

Proof: Let us consider

$SMSE\left({\hat{\gamma }}_{\left(i\right)}\right)-SMSE\left({\hat{\gamma }}_{\left(j\right)}\right)=tr\left(MSEM\left({\hat{\gamma }}_{\left(i\right)}\right)-MSEM\left({\hat{\gamma }}_{\left(j\right)}\right)\right)$

Using (4.1) we can write

$SMSE\left({\hat{\gamma }}_{\left(i\right)}\right)-SMSE\left({\hat{\gamma }}_{\left(j\right)}\right)=tr\left({\text{Δ}}_{\left(i,j\right)}\right)=tr(D_{\left(i,j\right)}+b_{\left(i\right)}{b^{\prime }}_{\left(i\right)}-b_{\left(j\right)}{b^{\prime }}_{(j))}$

Then ${\hat{\gamma }}_{\left(j\right)}$ is superior to ${\hat{\gamma }}_{\left(i\right)}$ if $SMSE\left({\hat{\gamma }}_{\left(i\right)}\right)-SMSE\left({\hat{\gamma }}_{\left(j\right)}\right)>0$ .

$SMSE\left({\hat{\gamma }}_{\left(i\right)}\right)-SMSE\left({\hat{\gamma }}_{\left(j\right)}\right)>0$ if and only if

$tr\left(D_{\left(i,j\right)}+b_{\left(i\right)}{b^{\prime }}_{\left(i\right)}-b_{\left(j\right)}{b^{\prime }}_{\left(j\right)}\right)>0$

$tr\left(D_{\left(i,j\right)}\right)>tr(b_{\left(j\right)}{b^{\prime }}_{\left(j\right)}-b_{\left(i\right)}{b^{\prime }}_{(i))}$

$1>\frac{tr\left(b_{\left(j\right)}{b^{\prime }}_{\left(j\right)}-b_{\left(i\right)}{b^{\prime }}_{\left(i\right)}\right)}{tr\left(D_{\left(i,j\right)}\right)}$

which completes the proof.

4.3. Mean Square Error Matrix (MSEM) Comparison of Generalized Predictors

If two generalized predictors ${\hat{y}}_{\left(i\right)}$ and ${\hat{y}}_{\left(j\right)}$ are given, the estimator ${\hat{y}}_{\left(j\right)}$ is said to be superior to ${\hat{y}}_{\left(i\right)}$ with respect to MSEM sense if and only if $MSEM\left({\hat{y}}_{\left(i\right)}\right)-MSEM\left({\hat{y}}_{\left(j\right)}\right)\ge 0$ .

Let us consider

$\begin{array}{l}MSEM\left({\hat{y}}_{\left(i\right)}\right)-MSEM\left({\hat{y}}_{\left(j\right)}\right)\\ =Z\left(MSEM\left({\hat{\gamma }}_{\left(i\right)}\right)-MSEM\left({\hat{\gamma }}_{\left(j\right)}\right)\right)Z^{\prime }-Z\left(Bias\left({\hat{\gamma }}_{\left(i\right)}\right)-Bias\left({\hat{\gamma }}_{\left(j\right)}\right)\right){\delta }^{\prime }\\ -\delta {\left(Bias\left({\hat{\gamma }}_{\left(i\right)}\right)-Bias\left({\hat{\gamma }}_{\left(j\right)}\right)\right)}^{\prime }{Z}^{\prime }\end{array}$

The following theorem can be stated for superiority of ${\hat{y}}_{\left(j\right)}$ over ${\hat{y}}_{\left(i\right)}$ with respect to the MSEM criterion.

Theorem 3: ${\hat{y}}_{\left(j\right)}$ is superior to ${\hat{y}}_{\left(j\right)}$ in MSEM sense when the regression model is misspecified due to excluding relevant variables if and only if $A\ge 0$ , $\theta \in \Re \left(A\right)$ and ${\theta }^{\prime }{A}^{-1}\theta \le 1$ , where

$A=Z{\text{Δ}}_{\left(i,j\right)}{Z}^{\prime }+\delta {\delta }^{\prime }+Z\left(b_{\left(i\right)}-b_{\left(j\right)}\right){\left(b_{\left(i\right)}-b_{\left(j\right)}\right)}^{\prime }{Z}^{\prime }$ , $\theta =\delta +Z\left(b_{\left(i\right)}-b_{\left(j\right)}\right)$ , $\Re \left(A\right)$ stands for column space of $A$ and $A^{-1}$ is an independent choice of g-inverse of $A$ .

Proof: Using (4.1) MSEM difference of the two generalized predictor can be written as

$MSEM\left({\hat{y}}_{\left(i\right)}\right)-MSEM\left({\hat{y}}_{\left(j\right)}\right)=Z{\text{Δ}}_{\left(i,j\right)}{Z}^{\prime }-Z\left(b_{\left(i\right)}-b_{\left(j\right)}\right){\delta }^{\prime }-\delta {\left(b_{\left(i\right)}-b_{\left(j\right)}\right)}^{\prime }{Z}^{\prime }$ (4.2)

After some straight forward calculation, equation (5.1) can be written as

$MSEM\left({\hat{y}}_{\left(i\right)}\right)-MSEM\left({\hat{y}}_{\left(j\right)}\right)=A-\theta {\theta }^{\prime }$

where $A=Z{\text{Δ}}_{\left(i,j\right)}{Z}^{\prime }+\delta {\delta }^{\prime }+Z\left(b_{\left(i\right)}-b_{\left(j\right)}\right){\left(b_{\left(i\right)}-b_{\left(j\right)}\right)}^{\prime }{Z}^{\prime }$ , $\theta =\delta +Z\left(b_{\left(i\right)}-b_{\left(j\right)}\right)$ and ${\text{Δ}}_{\left(i,j\right)}=D_{\left(i,j\right)}+b_{\left(i\right)}{b^{\prime }}_{\left(i\right)}-b_{\left(j\right)}{b^{\prime }}_{\left(j\right)}$ .

Due to Lemma 1 (see Appendix), $MSEM\left({\hat{y}}_{\left(i\right)}\right)-MSEM\left({\hat{y}}_{\left(j\right)}\right)$ is non-negative definite matrix if and only if $A\ge 0$ , $\theta \in \Re \left(A\right)$ and ${\theta }^{\prime }{A}^{-1}\theta \le 1$ , where $A=Z{\text{Δ}}_{\left(i,j\right)}{Z}^{\prime }+\delta {\delta }^{\prime }+Z\left(b_{\left(i\right)}-b_{\left(j\right)}\right){\left(b_{\left(i\right)}-b_{\left(j\right)}\right)}^{\prime }{Z}^{\prime }$ , $\theta =\delta +Z\left(b_{\left(i\right)}-b_{\left(j\right)}\right)$ , $\Re \left(A\right)$ stands for column space of $A$ and $A^{-1}$ is an independent choice of g-inverse of $A$ , which completes the proof.

Note that, obviously the conditions derived under Theorem 1 are sufficient for $A\ge 0$ . Consequently we may say that there are situations where ${\hat{y}}_{\left(j\right)}$ is superior to ${\hat{y}}_{\left(i\right)}$ in MMSE sense.

4.4. Scalar Mean Square Error (SMSE) Comparison of Generalized Predictors

Using (4.2) SMSE difference of the two generalized predictor can be written as

$\begin{array}{l}SMSE\left({\hat{y}}_{\left(i\right)}\right)-SMSE\left({\hat{y}}_{\left(j\right)}\right)=tr\left(MSEM\left({\hat{y}}_{\left(i\right)}\right)-MSEM\left({\hat{y}}_{\left(j\right)}\right)\right)\\ =tr\left(Z{\text{Δ}}_{\left(i,j\right)}{Z}^{\prime }-Z\left(b_{\left(i\right)}-b_{\left(j\right)}\right){\delta }^{\prime }-\delta {\left(b_{\left(i\right)}-b_{\left(j\right)}\right)}^{\prime }{Z}^{\prime }\right)\end{array}$

The following theorem can be stated for superiority of ${\hat{y}}_{\left(j\right)}$ over ${\hat{y}}_{\left(i\right)}$ with respect to the SMSE criterion.

Theorem 4: ${\hat{y}}_{\left(j\right)}$ is superior to ${\hat{y}}_{\left(i\right)}$ in SMSE sense when the regression model is misspecified due to excluding relevant variables if and only if

$\frac{tr\left(Z\left(b_{\left(i\right)}-b_{\left(j\right)}\right){\delta }^{\prime }+\delta {\left(b_{\left(i\right)}-b_{\left(j\right)}\right)}^{\prime }{Z}^{\prime }\right)}{tr\left(Z{\text{Δ}}_{\left(i,j\right)}{Z}^{\prime }\right)}<1.$

Proof: ${\hat{y}}_{\left(j\right)}$ is superior to ${\hat{y}}_{\left(i\right)}$ if $SMSE\left({\hat{y}}_{\left(i\right)}\right)-SMSE\left({\hat{y}}_{\left(j\right)}\right)>0$ .

$SMSE\left({\hat{y}}_{\left(i\right)}\right)-SMSE\left({\hat{y}}_{\left(j\right)}\right)>0$ if and only if

$tr\left(Z{\text{Δ}}_{\left(i,j\right)}{Z}^{\prime }-Z\left(b_{\left(i\right)}-b_{\left(j\right)}\right){\delta }^{\prime }-\delta {\left(b_{\left(i\right)}-b_{\left(j\right)}\right)}^{\prime }{Z}^{\prime }\right)>0$

$tr\left(Z{\text{Δ}}_{\left(i,j\right)}{Z}^{\prime }\right)>tr\left(Z\left(b_{\left(i\right)}-b_{\left(j\right)}\right){\delta }^{\prime }+\delta {\left(b_{\left(i\right)}-b_{\left(j\right)}\right)}^{\prime }{Z}^{\prime }\right)$

$1>\frac{tr\left(Z\left(b_{\left(i\right)}-b_{\left(j\right)}\right){\delta }^{\prime }+\delta {\left(b_{\left(i\right)}-b_{\left(j\right)}\right)}^{\prime }{Z}^{\prime }\right)}{tr\left(Z{\text{Δ}}_{\left(i,j\right)}{Z}^{\prime }\right)}$

which completes the proof.

Based on Theorem 1, Theorem 2, Theorem 3 and Theorem 4 we can obtain the corresponding results for each of the biased estimators and respective predictors by plugging the values for $G_{\left(i\right)}$ , $G_{\left(j\right)}$ , $D_{\left(i,j\right)}$ , $b_{\left(i\right)}$ and $b_{\left(j\right)}$ . The results are summarized in Tables A2-A6 in the Appendix.

5. Illustration of Theoretical Results

5.1. Numerical Example

To illustrate our theoretical results, we consider a dataset which gives total National Research and Development Expenditures―as a Percent of Gross National Product by Country: 1972-1986. It represents the relationship between the dependent variable $Y$ the percentage spent by the United States and the four other independent variables $X_1$ , $X_2$ , $X_3$ and $X_4$ . The variable $X_1$ represents the percent spent by former Soviet Union, $X_2$ that spent by France, $X_3$ that spent by West Germany, and $X_4$ that spent by the Japan. The data was discussed in Gruber [19] , and the data has been analysed by Akdeniz and Erol [20] , Li and Yang [21] and among others. Now we assemble the data as follows:

$X=\left(\begin{array}{ccccc}1& 1.9& 2.2& 1.9& 3.7\\ 1& 1.8& 2.2& 2.0& 3.8\\ 1& 1.8& 2.4& 2.1& 3.6\\ 1& 1.8& 2.4& 2.2& 3.8\\ 1& 2.0& 2.5& 2.3& 3.8\\ 1& 2.0& 2.6& 2.4& 3.7\\ 1& 2.1& 2.6& 2.6& 3.8\\ 1& 2.2& 2.6& 2.6& 4.0\\ 1& 2.3& 2.8& 2.8& 3.7\\ 1& 2.3& 2.7& 2.8& 3.8\end{array}\right)$ , $y=\left(\begin{array}{c}2.3\\ 2.2\\ 2.2\\ 2.3\\ 2.4\\ 2.5\\ 2.6\\ 2.6\\ 2.7\\ 2.7\end{array}\right)$

Note that the eigenvalues of the $X^{\prime }X$ are 312.932, 0.754, 0.045, 0.037, 0.002, the condition number is 299 and Variance Inflation Factor (VIF) values are 6.91, 21.58, 29.75, 1.79. Since condition number is greater than 100 and first three VIF values are greater than 5, which implies the existence of serious multicollinearity in the data set.

After the standardization of the data, the corresponding OLS estimator is

$\beta =b_s={\left({X^{\prime }}_s{X}_s\right)}^{-1}{X^{\prime }}_s{y}_s={\left(0.6402,-0.1179,0.4733,0.0139\right)}^{\prime }$

For the standardized data (since there are ten observations and four parameters), we obtain

${\hat{\sigma }}_s^2=\frac{{\left(y_s-X_s{b}_s\right)}^{\prime }\left(y_s-X_s{b}_s\right)}{10-4}=0.003932$

Table 1 shows the estimated SMSE values of OLSE, RE, AURE, LE, AULE, PCRE, r-k class estimator and r-d class estimator for the regression model when $\left(l,p\right)=\left(4,0\right)$ , $\left(l,p\right)=\left(3,1\right)$ , and $\left(l,p\right)=\left(2,2\right)$ with respect to shrinkage parameters (k/d), where $l$ denotes the number of variable in the model and p denotes the number of misspecified variables.

Table 2 shows the estimated SMSE values of the predictor of OLSE, RE, AURE, LE, AULE, PCRE, r-k class estimator and r-d class estimator for the regression

Table 1. Estimated SMSE values of the estimators.

From Table 1, it can be observed that the minimum SMSE of the estimators depends on the values of shrinkage parameters and the level of misspecification, which is agreed with our theoretical findings.

Table 2. Estimated SMSE values of the predictors.

According to Table 2, it can be observed that the predictors behave differently than the respective estimators, which is also agreed with our theoretical findings.

model when $\left(l,p\right)=\left(4,0\right)$ , $\left(l,p\right)=\left(3,1\right)$ , and $\left(l,p\right)=\left(2,2\right)$ for some selected shrinkage parameters (k/d). For simplicity we choose shrinkage parameter values k and d in the range (0, 1).

5.2. Monte Carlo Simulation Study

For further clarification, a Monte Carlo simulation study is done under different levels of misspecification using R 3.2.5. Following McDonald and Galarneau [22] , we can get explanatory variables as follows:

$x_{ij}={\left(1-{\rho }^2\right)}^{1/2}{z}_{ij}+\rho z_{i,p+1};i=1,2,\cdots ,n;j=1,2,\cdots ,m\text{.}$

where $z_{ij}$ is an independent standard normal pseudo random number, and $\rho$ is specified so that the theoretical correlation between any two explanatory variables is given by ${\rho }^2$ . A dependent variable is generated by using the equation

$y_i={\beta }_1{x}_{i1}+{\beta }_2{x}_{i2}+{\beta }_3{x}_{i3}+{\beta }_4{x}_{i4}+{\beta }_5{x}_{i5}+{\epsilon }_i;i=1,2,\cdots ,n\text{.}$

where ${\epsilon }_i$ is a normal pseudo random number with mean zero and variance one. In this study, we choose $\beta =\left({\beta }_1,{\beta }_2,{\beta }_3,{\beta }_4,{\beta }_5\right)$ as the normalized eigenvector corresponding to the largest eigenvalue of $X^{\prime }X$ for which ${\beta }^{\prime }\beta =1$ . We consider the following set up to investigate the effects of different degrees of multicollinearity on the estimators:

$\rho =0.9$ , condition number = 6.06 and VIF = (4.84, 4.83, 4.82, 4.81, 4.87)

$\rho =0.99$ , condition number = 20.12 and VIF = (46.09, 46.12, 46.02, 45.97, 46.56)

$\rho =0.999$ , condition number = 64 and VIF = (458.3, 459.2, 458.1, 457.8, 463.4)

Three different sets of observations are considered by selecting $\left(l,p\right)=\left(5,0\right)$ , $\left(l,p\right)=\left(4,1\right)$ and $\left(l,p\right)=\left(3,2\right)$ when $n=50$ , where $l$ denotes the number of variable in the model and p denotes the number of misspecified variables. For simplicity, we select values k and d in the range $\left(0,1\right)$ .

The simulation is repeated 2000 times by generating new pseudo random numbers and the simulated SMSE values of the estimators and predictors are obtained using the following equations:

$SMSE\left({\hat{\gamma }}_{\left(j\right)r}\right)=\frac{1}{2000}\underset{r=1}{\overset{2000}{{\displaystyle \sum }}}tr\left(MSEM\left({\hat{\gamma }}_{\left(j\right)r}\right)\right)$

and $SMSE\left({\hat{y}}_{\left(j\right)r}\right)=\frac{1}{2000}\underset{r=1}{\overset{2000}{{\displaystyle \sum }}}tr\left(MSEM\left({\hat{y}}_{\left(j\right)r}\right)\right)$ respectively.

Tables 3-5 are showing the estimated SMSE values of the estimators for the regression model when $\left(l,p\right)=\left(5,0\right)$ , $\left(l,p\right)=\left(4,1\right)$ and $\left(l,p\right)=\left(3,2\right)$ , and $\rho =0.9$ , $\rho =0.99$ and $\rho =0.999$ for the selected values of shrinkage parameters (k/d), respectively. Tables 6-8 are showing the corresponding estimated SMSE values of the predictors for the regression model, respectively.

From Tables 3-8, we can summarise the results as shown in Table 9.

6. Conclusions

In this study, a common form of superiority conditions were obtained for comparison among the biased estimators (RE, AURE, LE, AULE, PCRE, r-k class estimator and r-d class estimator) and their predictors by using a generalized form for the misspecified linear regression model when explanatory variables are multicollinearity. Furthermore, the theoretical findings were analyzed by using a numerical example and a Monte Carlo simulation study.

The simulation study shows that the LE and RE outperform the other estimators when weak multicollinearity exist, and RE, r-k class and r-d class estimators

Table 3. Estimated SMSE values of the estimators when $n=50$ and $\rho =\mathrm{0.9.}$

According to Table 3, it can be observed that, ${\hat{\gamma }}_d$ is superior over other estimators when $k\le 0.4$ and ${\hat{\gamma }}_k$ is superior over other estimators when $k>0.4$ under diferent levels of misspecifications for the weak multicollinearity.

Table 4. Estimated SMSE values of the estimators when $n=50$ and $\rho =\mathrm{0.99.}$

According to Table 4, it can be observed that, ${\hat{\gamma }}_{rd}$ is superior over other estimators when $k\le 0.4$ and ${\hat{\gamma }}_{rk}$ is superior over other estimators when $k>0.4$ under different levels of misspecifications for the moderated multicollinearity.

Table 5. Estimated SMSE values of the estimators when $n=50$ and $\rho =\mathrm{0.999.}$

According to Table 5, it can be observed that, ${\hat{\gamma }}_{rd}$ is superior over other estimators when $k\le 0.2$ under all level of misspecifications, ${\hat{\gamma }}_{rk}$ is superior over other estimators when $0.2<k\le 0.5$ and ${\hat{\gamma }}_k$ is superior over other estimators when $k>0.5$ under $\left(l,p\right)=\left(5,0\right)$ and $\left(l,p\right)=\left(4,1\right)$ , and ${\hat{\gamma }}_{rk}$ is superior over other estimators when $0.2<k\le 0.7$ and ${\hat{\gamma }}_k$ is superior over other estimators when $k>0.7$ under $\left(l,p\right)=\left(3,2\right)$ for high multicollinearity.

Table 6. Estimated SMSE values of the predictors when $n=50$ and $\rho =\mathrm{0.9.}$

According to Table 6, it can be observed that, ${\hat{y}}_d$ is superior over other estimators when $k\le 0.4$ and ${\hat{y}}_k$ is superior over other estimators when $k>0.4$ under different levels of misspecifications.

Table 7. Estimated SMSE values of the predictors when $n=50$ and $\rho =\mathrm{0.99.}$

According to Table 7, it can be observed that, ${\hat{y}}_d$ is superior over other estimators when $k\le 0.4$ and ${\hat{y}}_k$ is superior over other estimators when $k>0.4$ under different levels of misspecifications.

Table 8. Estimated SMSE values of the predictors when $n=50$ and $\rho =\mathrm{0.999.}$

According to Table 8, it can be observed that, ${\hat{y}}_d$ is superior over other estimators when $k\le 0.2$ and ${\hat{y}}_k$ is superior over other estimators when $k>0.2$ under different levels of misspecifications.

Table 9. Shrinkage parameter ranges for superior estimators and predictors.

outperform the other estimators when moderated and high multicollinearity exist for selected values of shrinkage parameters, respectively. It can also be noted that, the predictors based on the LE and RE are always superior to the other predictors for selected values of shrinkage parameters when multicollinearity exists among explanatory variables.

One of the limitation of this study is that we assume the error variance is equal for all models even when the variables are omitted from the model.

Appendix

Lemma 1: (Baksalary and Kala [23] )

Let $B\ge 0$ of type $n\times n$ matrix, $b$ is a $n\times 1$ vector and $\lambda$ is a positive real number. Then the following conditions are equivalent.

i) $\lambda B-b{b}^{\prime }\ge 0$

2) $B\ge 0$ , $b\in \Re \left(B\right)$ and $b^{\prime }{B}^{-1}b\le 1$ , where $\Re \left(B\right)$ stands for column space of B and $B^{-1}$ is a independent choice of g-inverse of B.

Lemma 2: (Trenkler and Toutenburg [24] )

Let ${\hat{\beta }}_1$ and ${\hat{\beta }}_2$ be two linear estimator of $\beta$ . Suppose that $D=D\left({\hat{\beta }}_1\right)$ $-D\left({\hat{\beta }}_2\right)$ is positive definite, then $\text{Δ}=MSE\left({\hat{\beta }}_1\right)-MASE\left({\hat{\beta }}_2\right)$ is non negative if and only if ${b^{\prime }}_2{\left(D+b_1{b^{\prime }}_1\right)}^{-1}{b}_2\le 1$ , where $D\left({\hat{\beta }}_j\right)$ , $MSE\left({\hat{\beta }}_j\right)$ and $b_j$ denote dispersion matrix, mean square error matrix and bias vector of ${\hat{\beta }}_j$ respectively, $j=1,2$ .

Lemma 3: (Wang et al. [25] )

Let $n\times n$ matrices $M>0,N\ge 0$ , then $M>N$ if and only if ${\lambda }_{*}<1$ , where ${\lambda }_{*}$ is the largest eigenvalue of the matrix $N{M}^{-1}$ .

Table A1. Expectation vector, Bias vector and Dispersion matrix of the estimators.

Table A2

Table A3

Table A4

Table A5

Table A6

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Sarkar, N. (1989) Comparisons among Some Estimators in Misspecified Linear Models with Multicollinearity. Annals of the Institute of Statistical Mathematics, 41, 717-724. https://doi.org/10.1007/BF00057737
[2]	Siray, G.ü. (2015) r-d Class Estimator under Misspecification. Communications in Statistics—Theory and Methods, 44, 4742-4756. https://doi.org/10.1080/03610926.2013.835421
[3]	Wu, J. (2016) Superiority of the r-k Class Estimator over Some Estimators in a Misspecified Linear Model. Communication in Statistics—Theory and Methods, 45, 1453-1458. https://doi.org/10.1080/03610926.2013.863934
[4]	Terasvirta, T. (1980) Linear Restrictions in Misspecified Linear Models and Polynomial Distributed Lag Estimation. Department of Statistics, University of Helsinki, Helsinki.
[5]	Mittelhammer, R.C. (1981) On Specification Error in the General Linear Model and Weak Mean Square Error Superiority of the Mixed Estimator. Communications in Statistics—Theory and Methods, 167-176. https://doi.org/10.1080/03610928108828027
[6]	Ohtani, K. and Honda, Y. (1984) On Small Sample Properties of the Mixed Regression Predictor under Misspecification. Communications in Statistics—Theory and Methods, 2817-2825. https://doi.org/10.1080/03610928408828863
[7]	Kadiyala, K. (1986) Mixed Regression Estimator under Misspecification. Economic Letters, 21, 27-30. https://doi.org/10.1016/0165-1765(86)90115-1
[8]	Trenkler, G. and Wijekoon, P. (1989) Mean Square Error Matrix Superiority of the Mixed Regression Estimator under Misspecification. Statistica, 49, 65-71.
[9]	Wijekoon, P. and Trenkler, G. (1989) Mean Square Error Matrix Superiority of Estimators under Linear Restrictions and Misspecification. Economics Letters, 30, 141-149. https://doi.org/10.1016/0165-1765(89)90052-9
[10]	Hubert, M. and Wijekoon, P. (2004) Superiority of the Stochastic Restricted Liu Estimator under Misspecification. Statistica, 64, 153-162.
[11]	Massy, F. (1965) Principal Components Regression in Exploratory Statistical Research. Journal of the American Statistical Association, 60, 234-266. https://doi.org/10.1080/01621459.1965.10480787
[12]	Hoerl, A. and Kennard, R. (1970) Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics, 12, 55-67. https://doi.org/10.1080/00401706.1970.10488634
[13]	Baye, R. and Parker, F. (1984) Combining Ridge and Principal Component Regression: A Money Demand Illustration. Communications in Statistics—Theory and Methods, 13, 197-205. https://doi.org/10.1080/03610928408828675
[14]	Singh, B., Chaubey, Y.P. and Dwivedi, T.D. (1986) An Almost Unbiased Ridge Estimator. The Indian Journal of Statistics, 48, 342-346.
[15]	Liu, K. (1993) A New Class of Biased Estimate in Linear Regression. Communications in Statistics—Theory and Methods, 22, 393-402. https://doi.org/10.1080/03610929308831027
[16]	Akdeniz, F. and Kaciranlar, S. (1995) On the Almost Unbiased Generalized Liu Estimator and Unbiased Estimation of the Bias and MSE. Communications in Statistics—Theory and Methods, 24, 1789-1797. https://doi.org/10.1080/03610929508831585
[17]	Kaciranlar, S. and Sakallioglu, S. (2001) Combining the Liu Estimator and the Principal Component. Communications in Statistics—Theory and Methods, 30, 2699-2705. https://doi.org/10.1081/STA-100108454
[18]	Xu, W. and Yang, H. (2011) On the Restricted r-k Class Estimator and the Restricted r-d Class Estimator in Linear Regression. Journal of Statistical Computation and Simulation, 81, 679-691. https://doi.org/10.1080/00949650903471023
[19]	Gruber, M. (1998) Improving Efficiency by Shrinkage: The James-Stein and Ridge Regression Estimators. CRC Press, New York.
[20]	Akdeniz, F. and Erol, H. (2003) Mean Squared Error Matrix Comparisons of Some Biased Estimators in Linear Regression. Communications in Statistics—Theory and Methods, 2, 389-2413. https://doi.org/10.1081/STA-120025385
[21]	Li, Y. and Yang, H. (2010) A New Stochastic Mixed Ridge Estimator in Linear Regression Model. Statistical Papers, 51, 315-323. https://doi.org/10.1007/s00362-008-0169-5
[22]	McDonald, G.C. and Galarneau, D.I. (1975) A Monte Carlo Evaluation of Some Ridge-Type Estimators. Journal of the American Statistical Association, 70, 407-416. https://doi.org/10.1080/01621459.1975.10479882
[23]	Baksalary, J. and Kala, R. (1983) Partial Orderings between Matrices One of Which Is of Rank One. Bulletin of the Polish Academy of Sciences Mathematics, 31, 5-7.
[24]	Trenkler, G. and Toutenburg, H. (1990) Mean Square Error Matrix Comparisons between Biased Estimators: An Overview of Recent Results. Statistical Papers, 31, 165-179. https://doi.org/10.1007/BF02924687
[25]	Wang, S., et al. (2006) Matrix Inequalities. 2nd Edition, Chinese Science Press, Beijing.