More on the Preliminary Test Stochastic Restricted Liu Estimator in Linear Regression Model

In this paper we compare recently developed preliminary test estimator called Preliminary Test Stochastic Restricted Liu Estimator (PTSRLE) with Ordinary Least Square Estimator (OLSE) and Mixed Estimator (ME) in the Mean Square Error Matrix (MSEM) sense for the two cases in which the stochastic restrictions are correct and not correct. Finally a numerical example and a Monte Carlo simulation study are done to illustrate the theoretical findings.


Introduction
To overcome the multicollinearity problem arises in the Ordinary Least Squares Estimation (OLSE) procedure, different methods have been proposed in the literature.One of the most important estimation methods is to consider biased estimators, such as the Ridge Estimator (RE) by Hoerl and Kennard [1], the Liu Estimator (LE) by Liu [2], and the Almost Unbiased Liu Estimator (AULE) by Akdeniz and Kaçiranlar [3].Alternative method to solve the multicollinearity problem is to consider parameter estimation with some restrictions on the unknown parameters, which may be exact or stochastic.When the stochastic restrictions are available in addition to sample model, Theil and Goldberger [4] introduced the Mixed Estimator (ME).Replacing OLSE by ME in the Liu Estimator, the Stochastic Restricted Liu Estimator (SRLE) has been proposed by Hubert and Wijekoon [5].
When different estimators are available, the preliminary test estimation procedure is adopted to select a suitable estimator, and it can also be used as another estimator with combining properties of both estimators.The preliminary test approach was first proposed by Bancroft [6], and then has been studied by many researchers, such as Judge and Bock [7], Wijekoon [8] and Arumairajan and Wijekoon [9].By combining OLSE and ME, the Ordinary Stochastic Preliminary Test Estimator (OSPE) was proposed by Wijekoon [8].Recently, Arumairajan and Wijekoon [9] introduced the Preliminary Test Stochastic Restricted Liu Estimator (PTSRLE) by combining the Stochastic Restricted Liu Estimator and Liu Estimator.In their study, they compared PTSRLE with SRLE by using the Mean Square Error Matrix (MSEM) and Scalar Mean Square Error (SMSE) criterions.
In this research we further compare the mean square error matrix of PTSRLE with OLSE and ME.The rest of the paper is organized as follows.The model specification and estimation are given in section 2. In section 3, the mean square error matrix comparisons between PTSRLE with OLSE and ME are performed.A numerical example and a Monte Carlo simulation are used to illustrate the theoretical findings in section 4, and in section 5 we state the conclusions.

Model Specification and Estimation
First we consider the multiple linear regression model where y is an n × 1 observable random vector, X is an n × p known design matrix of rank p, β is a p × 1 vector of unknown parameters and ε is an n × 1 vector of disturbances.
In addition to sample model ( 1), let us be given some prior information about β in the form of a set of m independent stochastic linear restrictions as follows; ( ) where r is an m × 1 stochastic known vector R is a m × p of full row rank m p ≤ with known elements, δ is non zero m × 1 unknown vector and υ is an m × 1 random vector of disturbances and Ω is assumed to be known and positive definite.Further it is assumed that υ is stochastically independent of ε. i.e. ( ) 0 The Ordinary Least Squares Estimator (OLSE) for model (1) and the Mixed Estimator (ME) (Theil and Goldberger [4]) due to a stochastic prior restriction (2) are given by ) ( ) respectively, where S X X ′ = .The expectation vector, and the mean square error matrix of β are given as ( ) and respectively.
The expectation vector, dispersion matrix, and the mean square error matrix of ˆME β are given as ( ) ) and respectively, where, ( ) . Note that when the stochastic restrictions are correct then ( ) = , and consequently the Mixed Estimator be- comes an unbiased estimator.
The Liu Estimator (Liu, [2]) is given as Replacing OLSE by ME in the Liu Estimator, Hubert and Wijekoon [5] introduced the Stochastic Restricted Liu Estimator (SRLE), and is given by ( ) when different estimators are available for the same parameter vector β in the linear regression model one must solve the problem of their comparison.Usually as a simultaneous measure of covariance and bias, the mean square error matrix of β is used, and is defined by Let us now turn to the question of the statistical evaluation of the compatibility of sample and stochastic information.The classical procedures is to test the hypothesis under linear model (1) and stochastic prior information (2).
The Ordinary Stochastic Preliminary Test Estimator (OSPE) of β (Wijekoon [8]) is defined as Further, we can write (15) as ) and are indicator functions which take the value one if F falls in the sub- scripted interval, and zero otherwise.

( )
The expectation vector, dispersion matrix, and the mean square error matrix of ˆOSPE β are derived by Wijekoon [8], and given by ( ) respectively, where, Pr for Recently, Arumairajan and Wijekoon [9] proposed the Preliminary Test Stochastic Restricted Liu Estimator (PTSRLE) by combining the Liu Estimator and Stochastic Restricted Liu Estimator, and is given by Note that the PTSRLE can be rewritten as follows: By using Equations ( 19), ( 20) and ( 21), Arumairajan and Wijekoon [9] derived the expectation vector, bias vector, dispersion matrix and mean square error matrix of PTSRLE as follows: respectively, where

Mean Square Error Matrix Comparisons
In this section we compare the PTSRLE with OLSE and ME in the mean square error matrix sense for the two cases in which the stochastic restrictions are correct and not correct.

Comparison between the PTSRLE and OLSE
The mean square error matrix difference between OLSE and PTSRLE can be written as ( ) where, ( ) ( ) is the value of ( ) 2) When the stochastic restrictions are not true (i.e.0 δ ≠ ), and if the maximum eigenvalue of To apply lemma 2 (Appendix), we have to show that ( )

S S I S dI S S dI S I S I S I S S I S dI S S dI S I d S I I d S S I
is positive definite matrix for all 0 1 d < < .According to lemma 1 (Appendix), To apply lemma 2 (Appendix) in order to prove that We have already proved that ( ) F > and 0 ξ > .Now according to lemma 3 (Appendix), ( ) ) ≤ .This completes the proof.

Comparison between the PTSRLE and ME
The mean square error matrix difference between ME and PTSRLE is ( ) where 2) When the stochastic restrictions are not true (i.e.0 δ ≠ ), and if the maximum eigen value of 1 3 1 D D − is less than one, the PTSRLE is superior to ME in the mean square error matrix sense if and only if ( ) 2) When stochastic restrictions are not true (i.e.0 δ ≠ ), ( ) ≥ lemma 4 (Appendix) can be used.To use this lemma first we prove that C is positive definite.According to lemma 3, C is a positive definite matrix if and only if the maximum eigenvalue of 1 3 1 D D − is less than one.By applying lemma we can prove ≤ .This completes the proof.According to theorem 1 and 2 it is clear that PTSRLE is superior to OLSE and ME under certain conditions.

Numerical Illustrations
In this section the comparison of PTSRLE with OLSE and ME are demonstrated using a numerical example, and a simulation study.

Numerical Example
To illustrate our theoretical result, we consider the data set on Total National Research and Development Expenditures as a percent of Gross National product due to Gruber [10].This data set is used by Akdeniz and Erol [11], Li and Yang [12] and Wu and Yang [13] to verify the theoretical results.Data shows Total National Research and Development Expenditures as a Percent of Gross National Product by Country: 1972Country: -1986.It represents the relationship between the dependent variable y the percentage spent by the United States and the four other independent variables 1 2 3 , , x x x and 4 x .The variable 1 x represents the percent spent by France, x that spent by West Germany, 3 x that spent by Japan, and 4 x that spent by the former Soviet Union.The data set is given below: 1.9 2.2 1. 9 The four column of the 10 × 4 matrix X comprise the data on 1 2 3 , , x x x and 4 x respectively, and y is the re- sponse variable.From this data, we obtain the following results: 1) The eigen values of X X ′ : 302.9626, 0.7283, 0.0447, 0.0345.2) The OLS estimator of ( ) : 0.6455, 0.0896, 0.1436, 0.1526 3) The OLS estimator of 2 2  : 0.0015 σ σ = .4) The condition number of X X ′ : 8781.53.The condition number implies the existence of multicollinearity in the data set.We consider the following stochastic restrictions (Li and Yang, [12]) Further the significance level is taken as α = 0.05. Figure 1 is drawn by using the SMSE obtained by using Equations ( 6), ( 9) and ( 27).
Based on Figure 1, we can say that the SMSE of PTSRLE is larger than the SMSE of both ME and OLSE when d is small.But the PTSRLE has the smallest SMSE than both ME and OLSE for 0.6029 0.9386 d < < . Moreover the ME is always superior to OLSE.

Monte Carlo Simulation
To illustrate the behavior of our proposed estimators, we perform the Monte Carlo Simulation study by considering different levels of multicollinearity.Following McDonald and Galarneau [14] we generate explanatory variables as follows: ( ) , , where ij z is an independent standard normal pseudo random number, and γ is specified so that the theoretical correlation between any two explanatory variables is given by 2 γ .A dependent variable is generated by using the equation.
where i ε is a normal pseudo random number with mean zero and variance 2 i σ .Newhouse and Oman [15] have noted that if the MSE is a function of 2 σ and β, and if the explanatory variables are fixed, then subject to the constraint 1 β β ′ = , the MSE is minimized when β is the normalized eigenvector corresponding to the largest eigenvalue of the X X ′ matrix.In this study we choose the normalized eigenvector corresponding to the largest eigenvalue of X X ′ as the coefficient vector β, n = 100, p = 4 and 2 1 i σ = .Four different sets of correlations are considered by selecting the values as γ = 0.9, 0.99, 0.999 and 0.9999, and the significance level is taken as α = 0.05.The condition numbers for γ = 0.9, 0.99, 0.999 and 0.9999 are 19.49,223.67, 2297.89 and 23137.88respectively.Figures 2-5 are drawn by using the SMSE obtained by using Equation ( 6), ( 9) and ( 27).According to Figure 2, the PTSRLE has the smallest SMSE than ME except d is small.However the OLSE has the smallest SMSE than ME and PTSRLE.From Figure 3, the SMSE of PTSRLE is larger than the SMSE of both OLSE and ME when d is small.From Figure 4 and Figure 5, we can notice that most of the cases, the PTSRLE has the smallest SMSE than ME and OLSE.

Conclusion
In this paper we have shown that the Preliminary Test Stochastic Restricted Liu Estimator is superior to Mixed Estimator and Ordinary Least Square Estimator in the mean square error matrix sense under certain conditions.From the simulation study and the numerical illustration we notice that the PTSRLE has the smallest SMSE than ME and OLSE when multicollinearity among the predictor variables is large.

1 )
When the stochastic restrictions are true (i.e.0 δ = ), the PTSRLE is superior to OLSE in the mean square error matrix sense if and only if * * 1

1 2 1 D
D − is less than one, the PTSRLE is superior to OLSE if and only if the equation (28) reduced to

* 1 D 2 )
is positive definite matrix since ( ) the PTSRLE is superior to OLSE in the mean square error matrix sense when stochastic restrictions are correct if and only if * * 1 If stochastic restrictions are not correct then ( )

1 ) 1 D
When the stochastic restrictions are true (i.e.0 δ = ), and if the maximum eigen value of * * 1 3 D − is less than one, the PTSRLE is superior to ME in the mean square error matrix sense if and only if * * 1

1 D 1 D
When stochastic restrictions are true then clearly a nonnegative matrix.We have already proved that * is positive definite matrix.Now according to lemma 3, only if the maximum eigen values of * * 1 3 D − is less than one.Now according to lemma 2, the PTSRLE is superior to ME if and only if * * 1