Improvement of the Preliminary Test Estimator When Stochastic Restrictions are Available in Linear Regression Model

Ridge type estimators are used to estimate regression parameters in a multiple linear regression model when multicolinearity exists among predictor variables. When different estimators are available, preliminary test estimation procedure is adopted to select a suitable estimator. In this paper, two ridge estimators, the Stochastic Restricted Liu Estimator and Liu Estimator are combined to define a new preliminary test estimator, namely the Preliminary Test Stochastic Restricted Liu Estimator (PTSRLE). The stochastic properties of the proposed estimator are derived, and the performance of PTSRLE is compared with SRLE in the sense of mean square error matrix (MSEM) and scalar mean square error (SMSE) for the two cases in which the stochastic restrictions are correct and not correct. Moreover the SMSE of PTSRLE based on Wald (WA), Likelihood Ratio (LR) and Lagrangian Multiplier (LM) tests are derived, and the performance of PTSRLE is compared using WA, LR and LM tests as a function of the shrinkage parameter d with respect to the SMSE. Finally a numerical example is given to illustrate some of the theoretical findings.


Introduction
A common problem in a multiple linear regression model is a multicolllinearity.Some biased estimators are proposed to solve this problem such as the Ordinary Ridge Estimator (ORE) by Hoerl and Kennard [1], the Restricted Ridge Estimator (RRE) by Sarkar [2], the Liu Estimator (LE) by Liu [3], the Restricted Liu Estimator (RLE) by Kaçiranlar, et al. [4] and the Stochastic Restricted Liu Estimator (SRLE) by Hubert and Wijekoon [5].When different estimators are available the preliminary test estimation procedure is adopted to select a suitable estimator.The preliminary test approach was first proposed by Bancroft [6] and then has been studied by many researchers, such as Judge and Bock [7], Wijekoon and Trenkler [8] and Saleh and Kibria [9].Later Kibria and Saleh [10] have discussed the performance of preliminary test ridge estimators based on WA [11], the LR [12] and the LM [13] tests.Then Yang and Xu [14] have introduced the preliminary test Liu estimators based on these three tests by combining the Restricted Liu Estimator (RLE) and the Liu Estimator.
In this paper, two ridge estimators, the Stochastic Restricted Liu Estimator and Liu Estimator are combined to define a new preliminary test estimator.The new PTSRLE is introduced and derives its stochastic properties in Section 2. The mean square error and scalar mean square error comparisons between PTSRLE and SRLE are carried out in Section 3. In Section 4 the SMSE of the PTSRLE based on WA, LR and LM tests are derived and the performance of the PTSRLE is compared using WA, LR and LM tests as a function of the shrinkage parameter d with respect to the Scalar Mean Square Error.Finally in Section 5, we illustrated these comparisons with a numerical example.
where is an n × 1 observable random vector, X is an n × p known design matrix of rank p,  is a p × 1 vector of unknown parameters and  is an n × 1 vector of disturbances.
In addition to sample Model (1), let us be given some prior information about  in the form of a set of m independent stochastic linear restrictions as follows; where is an m × 1 stochastic known vector is a m × p of full row rank with known elements,  is non zero m × 1 unknown vector and  is an m × 1 random vector of disturbances and is assumed to be known and positive definite.Further it is assumed that Let us now turn to the question of the statistical evaluation of the compatibility of sample and stochastic information.The classical procedures is to test the hypothesis 0 : 0 agains under linear Model (1) and stochastic prior information (2).The Ordinary Least Squares Estimator (OLSE) for the Model (1) and mixed estimator [15] due to a stochastic prior restriction (2) are given by  

) respectively, where
The Ordinary Stochastic Pre Test Estimator (OSPE) of  [8] is defined as Further, we can write (5) as follows where, which has a non-central , ,   D  is the dispersion matrix and where denotes the bias vector.We recall that the Scalar Mean Square Error and stochastic restricted Liu estimator (11) are combined to define the new preliminary test estimator (Preliminary Test Stochastic Restricted Liu Estimator (PTSRLE)) as (12) where, is the shrinkage parameter.with  Then we can write (12) as follows Wijekoon [8] derived the stochastic properties of OSPE.By using those results the expectation vector, bias vector, dispersion matrix, MSEM and SMSE of can be shown as follows and Hubert and Wijekoon [5] have given the MSE and SMSE for SRLE as Now we will see some properties of PTSRLE  Note that the PTSRLE reduces to the OSPE when which is the MSE matrix of Liu estimator.

  
, and hence from ( 17), the MSE matrix of the PTSRLE tends towards that of the LE.

Performance of the Proposed Estimator
In this section, we will compare the PTSRLE with the SRLE in the sense of mean square error matrix and scalar mean square error when stochastic restrictions are correct and not correct.
Definition: (MSEM Superiority of Estimators) 2 Let two alternative estimators 1  and  of  be given.Then 2 ˆ1  is said to be superior to  with respect to the MSEM criterion if and only if

Comparison between the PTSRLE and SRLE under MSE Criterion
In this subsection, we will compare the PTSRLE with SRLE under MSE criterion when the stochastic restrictions are correct and not correct.
Consider the MSE difference between the PTSRLE and SRLE, where,

Theorem 3.1:
1) If the stochastic restrictions are true (i.e.,   ); the SRLE is always superior to the PTSRLE in the mean squared error matrix sense.
2) Under the assumption   , the SRLE is not worse than the PTSRLE if and only if: in the mean square error matrix sense when the stochastic restriction are not true (i.e., 0   ).Here denotes the column space of the corresponding matrix.
) is nonnegative definite.Since is nonnegative definite, we can apply the lemma of [16] (see Appendix) to analyze the MSE matrix superiority of According to [17] (Theorem A.76, p. 514) we can derive the generalized inverse of as .
Using ( 24) and ( 25) we can easily prove that To establish condition (1) in the lemma (see Appendix), we find .
Hence, according to the lemma the mean square error matrix difference This completes the proof of theorem.

Comparison between the PTSRLE and SRLE under SMSE Criterion
In this subsection, we will compare the PTSRLE with the SRLE under SMSE criterion when stochastic restricttions are correct and not correct.
If the stochastic restrictions are correct then , and consequently the SMSE difference between and , and consequently since the matrix is positive definite, there exist an orthogonal matrix and a positive definite diagonal matrix , with .Then the SMSE difference between SRLE and PTSRLE can be written as where, is the diagonal element of the matrix Now we summarize our findings: respectively [18].

H , the It's known that under the null hypothesis
Copyright © 2013 SciRes.OJS three test statistics have the same asymptotic chi-square distribution with degrees of freedom [18].When the exact distribution is approximated by the asymptotic chisquare distribution, the critical value for an α-level test of o m H is approximated by the central chi-square critical value m   2   for large sample tests.This asymptotic chi-square distribution has wide application in the field of Econometrics.Based on the above tests, the PTSRLE takes the form [10] as is the upper percentiles of the central  distribution with m degrees of freedom.By using the equation in ( 18), now we can obtain the SMSE of the PTSRLE based on WA, LR and LM tests. where, , and takes the value for WA, LR and LM tests as We consider the SMSE difference between WA and LR tests of the PTSRLE where: where: Case I: If the stochastic restrictions are true then δ = 0.
Case II: If the stochastic restrictions are not true then  . We can rewrite the SMSE difference in (31) as follows 2 We can rewrite the SMSE difference in (32) as follow: (2 ) 2 Therefore, the SMSE difference in (36) is nonnegative definite if , where Now the performance of the PTSRLE estimator based on WA, LR and LM tests are compared with respect to the SMSE, and the following theorem can be stated.

Theorem 4.1:
1) The stochastic restrictions are true (i.e.δ = 0); then 2) The stochastic restrictions are not true (i.e.  ) then a) If then where, 1 , , 3 and 4 are given in Equations (34), ( 35), (37) and (38), respectively.* 2 d From Theorem 4.1(2b) and according to [14] we can say that when d is small, WA test has the smallest SMSE than the other tests.Similarly according to the results stated in (2a), the LM test has the smallest SMSE than the other tests when becomes large.

observations and
 predictor variables.Since the regressor matrix X does not include a column of ones a homogeneous multiple linear regression, Model (1) without intercept is fitted to the data.
The ordinary least square estimator of regression coefficient  is   (see [20,21]).
Figures 1 and 2 are drawn by using the SMSE given in Equations ( 18) and (20) for different values selected from (0, 1).
According to the Figures 1 and 2, we can conclude that when is small the PTSRLE has the smallest SMSE value than the SRLE, OSPE and OLSE.From Figures 3 and 4, we can notice that when is small, the WA test has the smallest SMSE than the other tests.When becomes large, the LM test has the smallest SMSE.Hence the data analysis supports the findings of this paper.

Conclusions
In this paper, we have introduced a new preliminary test estimator in a multiple linear regression model.When is small, the PTSRLE based on WA test has the smallest SMSE than the other tests.When becomes large, the PTSRLE based on LM test has the smallest SMSE.Moreover, for certain cases (Figures 1 and 2) the proposed estimator has the smallest SMSE.The results of

Appendix
Lemma: (Baksalary and Trenkler, [16]) Let be a nonnegative definite matrix and 1 c , 2 be linearly independent vectors.Furthermore for some generalized inverse of , let which take the value one if F falls in the subscripted interval and zero otherwise.α-level critical value from the central F distribution .When different estimators are available for the same parameter vector  in the linear regression model one must solve the problem of their comparison.Usually as a simultaneous measure of covariance and bias, the mean square error matrix is used, and is defined by If the stochastic restrictions are correct then   E r , and consequently the Equation (22) reduced to

.
Now we consider the SMSE difference between LR and LM tests of the PTSRLE

Figures 3 and 4
are drawn by using the SMSE given in Equation (30) for different values selected from (0, 1).

PTSRLE Based on WA, LR and LM Tests
* 0 d d   * d 4.
[19]llustrate our theoretical results, we consider the following data set on Portland cement originally due to Woods, Steinour and Starke[19].This data set came from an experimental investigation of the heat evolved during the setting and hardening of Portland cements of varied composition and the dependence of this heat on the percentages of four compounds in the clinkers from which the cement was produced.