Goodness of Fit Test Based on BLUS Residuals for Error Distribution of Regression Model

Jianxin Zhao¹, Xinmin Li²

¹Navy Submarine Academy, Qingdao, China.
²School of Mathematics and Statistics, Qingdao University, Qingdao, China.

DOI: 10.4236/am.2022.138042                

Abstract

The error distribution testing plays an important role in linear regression as distribution misspecification seriously affects the validity and efficiency of regression analysis. The least squares (OLS) residuals are often used to construct test statistics; in order to overcome the non-independent and identically residuals, the best linear unbiased scale (BLUS) residuals are applied in this paper, which, unlike OLS residuals, the residuals vector is identically and independently distributed. Based on the BLUS residuals, a new test statistic is constructed by using the sample random distance between sample quantile and quasi sample quantile derived from the null distribution, and the goodness-of-fit test of error distribution in the linear model is studied. The powers of the new tests under certain alternatives are examined. They are more powerful tests for the hypotheses concerned.

Keywords

Goodness of Fit, Cramér-Von Mises Statistic, Kolmogorov-Smirnov Statistic, Anderson-Darling Statistic, Sample Quantiles, Stochastic Sample Quantiles

1. Introduction

Parametric and nonparametric regression models are widely applied to the fields such as biology, chemistry and economics. The general form is often written as

$Y=m\left(x\right)+\epsilon \mathrm{,}$ (1)

where m is the regression function, the error $\epsilon$ satisfies $\text{E}\left(\epsilon \right)=0$ and $\text{E}\left({\epsilon }^2\right)>0$, x are explanatory variables. The following general assumption is that the errors corresponding to different observations are independent and identically distributed. The most popular model is the linear model, that is, $m\left(x\right)=x^{\text{T}}\beta$, $\beta$ assumed as unknown parameters, and the error terms of the model are distributed according to the normal distribution. Under the assumption, the linear model has been attracting practical and theoretical workers because of its simplicity and validity.

Obviously, it is important to test whether the error is distributed as the assumed distribution before using the linear model to analyze the data under a certain assumption of the error distribution. On the other hand, even in general nonparametric models, additional knowledge of the distribution of errors can also improve the effectiveness of statistical analysis. For example, under the assumption of normal error, accurate or optimal tests can be obtained in many cases. A typical example is the goodness-of-fit test of regression function (See ref. [1] [2] [3]).

From the above analysis and the existing literature, it is easy to see that there are two kinds of important goodness of fit tests to the model. One is the goodness-of-fit test of the error distribution; the other is about the goodness of fit test of the regression function. That is to test the hypothesis:

$H\mathrm{<mo>\; :\; </mo>}\epsilon ~N\left(\mathrm{0,}{\sigma }^2\right)$ (2)

and

$H\mathrm{<mo>\; :\; </mo>}m\left(x\right)\in \mathcal{M}\mathrm{,}$ (3)

where $\mathcal{M}$ is a class of regression functions with certain properties.

There are many kinds of literatures about the two kinds of tests mentioned above, especially about the second kind of tests. For example, Of the first kind of test are ref. [4] [5] [6], etc., the second kind of test are ref. [7] - [17], etc.

For Linear regression models, there are usually two kinds of residual construction tests. One is based on the ordinary least square (OLS) residuals, under the normal assumption, the residual vector of OLS has a singular normal distribution, and the components of a vector are no longer independently identically distributed. The other is best linear unbiased scale residuals (BLUS) (See ref. [18]). It is a kind of residuum given by Theil in consideration of such a fact where the residuals are neither independent nor identical, even if the error distribution is independent and identical. Using OLS residuals directly to test the sequence independence or the same variance is impossible (See ref. [19]). However, the BLUS residual vector, under normal assumptions, is different from the OLS residual. There is a non-singular normal distribution, and the components of the vector are identically and independently distributed. It is a fact that most of the existing literature uses the OLS residuals to construct the test. But as you can see from the above analysis, using residuals as a new sample, it is not natural to construct a test with the existing test, such as the Shapiro-Wilk test, because the test of the original structure is based on independent samples from the same distribution, at the same time, it was proved that the skewness and Kurtosis of OLS residuals cannot exceed the skewness and Kurtosis of the error term (See ref. [20]). So, if the distribution of error terms is not normal, the distribution of OLS residuals is always close to the normal form, not the probability distribution of error terms. This shows that when the null hypothesis is not true, any normal test using the OLS residuals directly seems to have a tendency not to reject the null hypothesis. In view of the above analysis, this paper uses BLUS residuals to construct test statistics.

This paper is organized as follows. Section 2 constructs the new tests. Power comparisons are given in Section 3. Simulated powers for the new statistics are tabulated in this section. Section 4 gives some comments.

2. Test Statistics Based on BLUS Residuals

Let’s have a linear model as follows:

$y=X\beta +\epsilon \mathrm{,}\text{E}\epsilon =0\mathrm{,}\text{E}\epsilon {\epsilon }^{\text{T}}={\sigma }^{2}I\mathrm{,}$ (4)

where $y$ is a $n\times 1$ random vector, $X$ is a non-random $n\times k$ matrix with a known rank of k, $\beta$ is a $k\times 1$ unknown parametric vector. For the above model, the Normality test of the error term distribution is important. Many scholars have studied this problem. The early literature includes ref. [20] [21] and so on. For the past twenty years, the two papers (See ref. [22] [23]) are based on Shapiro & Wilk test statistics (See ref. [24]) to construct tests using OLS residuals. Furthermore, ref. [25] constructs test statistics by the normalized residuals based on Shapiro & Wilk test statistics. While ref. [26] using Bootstrap method and based on the process of empirical residuals, constructs a new test statistic by using KS and AD statistics. Recently, ref. [27] proposed several tests based on partial sums of residuals where the test statistics are based on sums of a subset of the (ordered and standardized) residuals (Also see ref. [28] [29]).

For a linear model (4), the least square estimate of the Regression Coefficient is $\hat{\beta }={\left(X^{\text{T}}X\right)}^{-1}{X}^{\text{T}}y$, using $e$ to represent the OLS residuals associated with the error vector $\epsilon$, we have

$e=y-X\hat{\beta }=\left(I-X{\left(X^{\text{T}}X\right)}^{-1}{X}^{\text{T}}\right)y=My=M\epsilon \mathrm{.}$ (5)

And from that, $\text{E}\epsilon =0\mathrm{,}\text{E}\epsilon {\epsilon }^{\text{T}}={\sigma }^{2}M$.

According to Theil (See ref. [18]), the BLUS residuals are obtained by the following steps:

First, select the smallest K elements in the main diagonal elements of the matrix $M$ and rearrange the observed value Y according to the position of the row in which the K elements are located. Might as well be, place them in the first K position (See ref. [30]). The original model is then divided into blocks:

$\left(\begin{array}{c}{y}_0\\ y_1\end{array}\right)=\left(\begin{array}{c}{X}_0\\ X_1\end{array}\right)\beta +\left(\begin{array}{c}{\epsilon }_0\\ {\epsilon }_1\end{array}\right)$ (6)

$M=\left[\begin{array}{cc}I-X_0{\left(X^{\text{T}}X\right)}^{-1}{X}_0^{\text{T}}& -X_0{\left(X^{\text{T}}X\right)}^{-1}{X}_1^{\text{T}}\\ -X_1{\left(X^{\text{T}}X\right)}^{-1}{X}_0^{\text{T}}& I-X_1{\left(X^{\text{T}}X\right)}^{-1}{X}_1^{\text{T}}\end{array}\right]\mathrm{,}$ (7)

where the $I$ in the upper left-hand corner in (7) is the $k\times k$ unit matrix, and the $I$ in the lower right-hand corner is the $\left(n-k\right)\times \left(n-k\right)$ unit matrix.

Secondly, compute the eigenvalues, denoted by $d_1^2\mathrm{,}\cdots \mathrm{,}{d}_k^2$, of the matrix $X_0{\left(X^{\text{T}}X\right)}^{-1}{X}_0^{\text{T}}$, and the corresponding eigenvectors by $q_1\mathrm{,}\cdots \mathrm{,}{q}_k$.

Finally, calculate the BLUS residuals $\hat{\epsilon }=e_1-X_1{X}_0^{-1}\left[{\displaystyle {\sum }_{i=1}^k}\frac{d_i}{1+d_i}{q}_i{q}_i^{\text{T}}\right]e_0$, here, $e_0\mathrm{,}{e}_1$ is the block residuals vector corresponding to the error term in (6). At this point, under the original assumption (2), the BLUS residual vector is

$\hat{\epsilon }~N\left(\mathrm{0,}{\sigma }^2{I}_{n-k}\right)\mathrm{.}$ (8)

The tests in this section and the next are constructed from this new sample. Denote ${\hat{\epsilon }}_{\left(1\right)}\mathrm{,}\cdots \mathrm{,}{\hat{\epsilon }}_{\left(m\right)}$ be an order statistic of BLUS residuals, $m=n-k$, $F_0\left(x\right)$ be the normal distribution.

BLUS residuals were mainly applied to multivariate models (See ref. [31]), the cusum and the cusum-of-squares tests (See ref. [19]) which have higher power than those based on the more popular recursive residuals for structural break, and the Dickey-Fuller unit root test which is based on BLUS residuals (See ref. [32]), etc.

The test statistics constructed in this paper and the competition test statistics available in the literature are described below.

The competition test statistics based on the empirical distribution function are KS statistics

${\hat{D}}_m=\sqrt{m}\underset{1\le i\le m}{\max }\left\{\left|\frac{i}{m}-F_0\left(\tilde{\epsilon }\right)\right|\mathrm{,}\left|\frac{i-1}{m}-F_0\left(\tilde{\epsilon }\right)\right|\right\}$ (9)

and AD statistics

${\hat{A}}_m^2=-\frac{1}{m}{\displaystyle \underset{i=1}{\overset{m}{\sum }}}\left[\left(2i-1\right)\log F_0\left(\tilde{\epsilon }\right)+\left(2m+1-2i\right)\log \left(1-F_0\left(\tilde{\epsilon }\right)\right)\right]-m\mathrm{,}$ (10)

here $\tilde{\epsilon }={\hat{\epsilon }}_{\left(i\right)}/\sqrt{S_m^2}$, since the mean of the original hypothesis is 0, the parameter estimation of the variance of the error is $S_m^2=1/m{\displaystyle \underset{i=1}{\overset{m}{\sum }}}{\hat{\epsilon }}_i^2$.

The test statistic based on sample order statistics or sample quantiles is Shapiro & Wilk test statistic (See ref. [33]):

$W=\frac{{\left({\displaystyle \underset{i=1}{\overset{m}{\sum }}}{a}_i{\hat{\epsilon }}_{\left(i\right)}\right)}^2}{m\cdot S_m^2}$ (11)

where $a={\left(a_1\mathrm{,}\cdots \mathrm{,}{a}_m\right)}^{\text{T}}=\frac{V^{-1}\text{E}\tilde{W}}{\sqrt{\text{E}{\tilde{W}}^{\text{T}}{V}^{-1}{V}^{-1}\text{E}\tilde{W}}}\mathrm{,}$ $V=\left(v_{ij}\right)=Var\left(\tilde{W}-\text{E}\tilde{W}\right)$, $\tilde{W}=\left({\tilde{W}}_{\left(1\right)},\cdots ,{\tilde{W}}_{\left(m\right)}\right)$ be the sample order statistics from $F_0(\cdot )$.

The test statistic constructed based on De Wet and Venter’ idea (See ref. [34]) is

${\hat{T}}_2={\displaystyle \underset{i=1}{\overset{m}{\sum }}}{\left(\frac{{\hat{\epsilon }}_{\left(i\right)}}{\hat{\theta }}-F_0^{-1}\left(\frac{i}{m+1}\right)\right)}^2,$ (12)

where $F_0^{-1}\left(u\right)=\inf \left\{x:F_0\left(x\right)\ge u\right\}$, $\hat{\theta }=\sqrt{S_m^2}$ be the estimation of variance of standard error.

Del Barrio, Couesta-Albertos, Matrán, Rodríguez-Rodríguez (See ref. [35]) proposed that the statistic constructed based on the distance of L₂-Wasserstein is

$\widehat{BCMR}=1-\frac{{\left({\displaystyle \underset{i\mathrm{<mo>\; =\; </mo>1}}{\overset{m}{\sum }}}{\hat{\epsilon }}_{\left(i\right)}{\displaystyle {\int }_{\left(i-1\right)/m}^{i/m}}{\Phi }^{-1}\left(t\right)\text{d}t\right)}^2}{S_m^2}\mathrm{,}$ (13)

Let $X_{\left(1\right)}\mathrm{,}\cdots \mathrm{,}{X}_{\left(m\right)}$ be the order statistics of $X_1\mathrm{,}\cdots \mathrm{,}{X}_m$ and $U_{\left(1\right)}\mathrm{,}\cdots \mathrm{,}{U}_{\left(m\right)}$ are the order statistics from $U\left(\mathrm{0,1}\right)$. If $X_1\mathrm{,}\cdots \mathrm{,}{X}_m$ is an iid sample from the continuous cumulative distribution function $F\left(x\right)$, the following equation holds (See ref. [36]).

$\left(X_{\left(1\right)}\mathrm{,}\cdots \mathrm{,}{X}_{\left(m\right)}\right)\stackrel{d}{=}\left(F^{-1}\left(U_{\left(1\right)}\right)\mathrm{,}\cdots \mathrm{,}{F}^{-1}\left(U_{\left(m\right)}\right)\right)\mathrm{,}$ (14)

where $\stackrel{d}{=}$ stands for equality in distribution.

Using this conclusion, Zhao (See ref. [37] [38]) thinks that if $X_1\mathrm{,}\cdots \mathrm{,}{X}_m$ doesn't come from the null distribution $F\left(x\right)$, (14) doesn’t hold and then there are differences between $\left(X_{\left(1\right)}\mathrm{,}\cdots \mathrm{,}{X}_{\left(m\right)}\right)$ and $\left(F^{-1}\left(U_{\left(1\right)}\right)\mathrm{,}\cdots \mathrm{,}{F}^{-1}\left(U_{\left(m\right)}\right)\right)$. The larger the differences are, the greater the evidence against the null hypothesis is. Using this idea, Zhao (See ref. [37]) constructed criteria to describe the differences based on random distance.

$ZR={\displaystyle \underset{i=1}{\overset{m}{\sum }}}{\left(X_{\left(i\right)}-F^{-1}\left(U_{\left(i\right)}\right)\right)}^2/{\sigma }_i^2$ (15)

In this paper, the null hypothesis is that the distribution function is a normal distribution function with only scale parameters. So (14) is reduced to

$X_{\left(i\right)}\stackrel{d}{=}\sigma Z_{\left(i\right)},i=1,\cdots ,m.$ (16)

And then, with respect to the parameter ${\sigma }^2$, take the smallest value of (15), it can be obtained that:

$ZR={\displaystyle \underset{i=1}{\overset{m}{\sum }}}{Z}_{\left(i\right)}^2+\frac{{\left[{\displaystyle {\sum }_{i=1}^m{X}_{\left(i\right)}{Z}_{\left(i\right)}}\right]}^2}{{\displaystyle {\sum }_{i=1}^m{X}_{\left(i\right)}^{\mathrm{<mtext>\; \hspace{0.17em}\; </mtext>\; 2}}}}$ (17)

where $Z_{\left(1\right)}\mathrm{,}\cdots \mathrm{,}{Z}_{\left(m\right)}$ is the order statistics for a sample of m size from the normal distribution. Note that (8) and above analysis, using ${\hat{\epsilon }}_{\left(i\right)}$ instead of $X_{\left(i\right)}$, we obtain the following test statistics in this paper.

$\widehat{ZR}={\displaystyle \underset{i=1}{\overset{m}{\sum }}}{Z}_{\left(i\right)}^2+\frac{{\left[{\displaystyle {\sum }_{i=1}^m{\hat{\epsilon }}_{\left(i\right)}{Z}_{\left(i\right)}}\right]}^2}{{\displaystyle {\sum }_{i=1}^m{\hat{\epsilon }}_{\left(i\right)}^{\mathrm{<mtext>\; \hspace{0.17em}\; </mtext>\; 2}}}},$ (18)

where $\widehat{ZR}$ is scale-invariant. For the same reason as in Zhao (See ref. [37]), we can take the q quantile and expectation of $\widehat{ZR}$ as the test statistics. Let $\widehat{Z{R}_q}$ and $\widehat{Z{R}_{\mu }}$ be the qth quantile and the expectation of $\widehat{ZR}$. Here we select only $q=0.05$, $q=0.50$, and $q=0.95$ as quantile statistics. Using BLUS residuals to solve the critical value of each test statistic is the same as Zhao’s (See ref. [37]) algorithm, which is omitted here.

3. Power Comparisons

In order to compare the power of the above tests, the following linear models are considered.

$y_i=1+2x_{1i}-3x_{2i}+{\epsilon }_i,i=1,2,\cdots ,n.$ (19)

The variable x is independent of ${\epsilon }_i$, which comes from a uniform distribution, these values are constant for a given sample size. The error variable $\epsilon$ follows an alternative distribution.

Many continuous alternative distributions are chosen which were used in the power studying by Shapiro et al. (See ref. [33]), Gan and Koehler (See ref. [39]) and Eva Krauczi (See ref. [40]).

The first group: alternative distributions are the non-normal distribution, such as the chi-square distribution with a degree of freedom of 3, denoted by ${\chi }^2\left(3\right)$ ; the exponential distribution with a mean of 1, denoted by Exp(1); the gamma distribution functions with shape parameters of 0.8 and 1.5, Gamma(0.8), Gamma(1.5), respectively; the double exponential distribution laplace(0,1); the log-normal distribution lognormal(0,1); the Cauchy distribution Cauchy(0,1); the student distribution with degree of freedom of 3, t(3); the unbounded Johnson’s distribution of the random variable sinh (Z), denoted as SU(0,1), where $Z~N\left(\mathrm{0,1}\right)$.

The second group: in heteroscedasticity case, the he model under consideration at this point is

$y_i=1+2x_{1i}-3x_{2i}+{\sigma }_i{\bar{\epsilon }}_i,i=1,2,\cdots ,n,$ (20)

where ${\sigma }_i$ be distributed as uniform distribution $U\left(\mathrm{0,50}\right)$, $i=1,2,\cdots ,n$. At the same time, the error variable $\bar{\epsilon }$ follows logistic(0,1), Cauchy(0,1), N(0,1) and Laplace(0,1), respectively.

The third group: the case of outlier with non-zero mean of regression error, such as

${\epsilon }_1~N\left(2,1\right),{\epsilon }_i\sim N\left(0,1\right),i=2,\cdots ,n,$

and

${\epsilon }_i~N\left(2,1\right),i=1,\cdots ,5,{\epsilon }_i~N\left(0,1\right),i=6,\cdots ,n,$

and also in high Leverage outlier case, like that

${\epsilon }_1~N\left(8,1\right),{\epsilon }_i~N\left(0,1\right),i=2,\cdots ,n,x_{11}=x_{21}=2.$

In this section, the selected test level is 5%, the sample size is $n=20,50$, and the empirical power of the test is based on 10,000 simulations. The results are placed in Table 1 and Table 2. From these two tables, you can see that:

Table 1. Powers for testing the error distribution is normal, against different non-normal alternative distributions, at significance level 5% and $n=20,n=50$.

^{a $G\left(0.8\right)\mathrm{,}G\left(1.5\right)$} : the Gamma distribution with shape parameter 0.8, 1.5. ^{b $F_1\left(x\right)$} : the LogNormal(0, 1) distribution. ^{c $F_3\left(x\right)$} : $X=\sinh \left(Z\right)$, $Z~N\left(0,1\right)$.

1) When the alternative distribution is a non-normal distribution function, the power of the test ${\widehat{ZR}}_{q=0.05}$ is significantly higher than that of other tests. When the capacity is 20, the power of ${\widehat{ZR}}_{\mu }$ and T₂ is similar. However, when the capacity is 50, for an asymmetric alternative distribution, BCMR, T₂, W and ${\widehat{ZR}}_{\mu }$ are not much different, for a symmetric alternative distribution, T₂ is the best, coming next is ${\widehat{ZR}}_{\mu }$. In short, Among the opponents of the new tests, the order of superiority and inferiority of the tests is T₂, BCMR, Shapiro-Wilk test, $A_n^2$ and $D_n$.

2) For an alternate distribution of $N\left(\mathrm{0,1}\right)$ the power of the test is approximately equal to the probability of the first type of error. As can be seen from Table 2, under the different sample sizes, all the tests make full use of the approximately 5% test level.

3) For the alternative distribution of the second group, the heteroscedasticity case, the power of ${\widehat{ZR}}_{q=0.05}$ test is significantly higher than the other tests, the

Table 2. Powers for testing the error distribution is normal, against Heteroscedasticity, outlier alternative distribution and normal alternative distributions, at significance level 5% and $n=20,n=50$.

^aThe error $\sigma \bar{\epsilon }$, $F_1\left(x\right)$ : $\bar{\epsilon }$ the Cauchy distribution; $F_2\left(x\right)$ : $\bar{\epsilon }$ the Normal distribution; $F_3\left(x\right)$ : $\bar{\epsilon }$ the Laplace distribution; $F_4\left(x\right)$ : $\bar{\epsilon }$ the Logistic distribution $\sigma ~U\left(0,50\right)$. ^{b $F_5\left(x\right)$} : ${\epsilon }_1~N\left(2,1\right)$, ${\epsilon }_i~N\left(0,1\right)$, $i=2,\cdots ,n$ ; $F_6\left(x\right)$ : ${\epsilon }_i~N\left(2,1\right)$, $i=1,\cdots ,5$, ${\epsilon }_i~N\left(0,1\right)$, $i=6,\cdots ,n$ ; $F_7\left(x\right)$ : ${\epsilon }_1~N\left(8,1\right)$, ${\epsilon }_i~N\left(0,1\right)$, $i=2,\cdots ,n$, $x_{11}=x_{12}=2$.

other results are similar to those in (1).

4) For the alternative distribution of the third group, where there are outliers. In the new test, ${\widehat{ZR}}_{q=0.95}$ performs best, but not much different from ${\widehat{ZR}}_{q=0.50}$, ${\widehat{ZR}}_{\mu }$. Compared with the comparative test, the difference is not significant, $A_n^2$ is slightly better, while ${\widehat{ZR}}_{q=0.05}$ performs worse.

4. Comments

Based on the BLUS residuals, using the difference between the residuals order statistics and the pseudo-random sample order statistics, the quantile-type and the conditional expectation-type test statistics are constructed, which are used in the error distribution Normality test of the linear regression model. The simulation results show that the power of the tests provided in this paper is better than some tests in the literature. Of course, T₂, BCMR and W are also good tests.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Eubank, R.L. and Hart, J.D. (1992) Testing Goodness-of-Fit in Regression via Order Selection Criteria. Annals of Statistics, 20, 1412-1425. https://doi.org/10.1214/aos/1176348775
[2]	Milhem, H. (2003) Testing the Goodness-of-Fit in a Gaussian Regression. Proceedings of the European Young Statisticians Meeting (EYSM), Ovronnaz, 21-26 September 2003, 101-111.
[3]	Young, S.G. and Bowman, A.W. (1995) Non-Parametric Analysis of Covariance. Biometrics, 51, 920-931. https://doi.org/10.2307/2532993
[4]	Durbin, J. (1973) Distribution Theory for Tests Based on the Sample Distribution Function. Society for Industrial and Applied Mathematics, Philadelphia. https://doi.org/10.1137/1.9781611970586
[5]	Durbin, J., Knott, M. and Taylor, C.C. (1975) Components of Cramérvon Mises statistics II. Journal of the Royal Statistical Society: Series B (Methodological), 37, 216-237. https://doi.org/10.1111/j.2517-6161.1975.tb01537.x
[6]	D’Agostino, R.B. and Stephens, M.A. (1986) Goodness-of-Fit Techniques. Marcel Dekker, Inc., New York.
[7]	Cox, D., Koh, E., Wahba, G. and Yandell, B.S. (1988) Testing the (Parametric) Null Model Hypothesis in (Semiparametric) Partial and Generalized Spline Models. Annals of Statistics, 16, 113-119. https://doi.org/10.1214/aos/1176350693
[8]	Gu, C. (1992) Diagnostics for Nonparametric Regression Models with Additive Terms. Journal of the American Statistical Association, 87, 1051-1058. https://doi.org/10.1080/01621459.1992.10476260
[9]	Eubank, R.L. and Hart, J.D. (1993) Commonality of Cusum, von Neumann and Smoothing-Based Goodness-of-Fit Tests. Biometrika, 80, 89-98. https://doi.org/10.1093/biomet/80.1.89
[10]	Eubank, R.L. and La Riccia, V.N. (1993) Testing for No Effect in Nonparametric Regression. Journal of Statistical Planning and Inference, 36, 1-14. https://doi.org/10.1016/0378-3758(93)90097-P
[11]	Stute, W. (1997) Nonparametric Model Checks for Regression. Annals of Statistics, 25, 613-641. https://doi.org/10.1214/aos/1031833666
[12]	Stute, W. and Gonzáilez Manteiga, W. (1996) NN Goodness-of-Fit Tests for Linear Models. Journal of Statistical Planning and Inference, 53, 75-92. https://doi.org/10.1016/0378-3758(95)00144-1
[13]	Stute, W., González Manteiga, W. and Presedo Quindimil, M. (1998) Bootstrap Approximations in Model Checks for Regression. Journal of the American Statistical Association, 93, 141-149. https://doi.org/10.1080/01621459.1998.10474096
[14]	Stute, W., Thies, S. and Zhu, L.X. (1998) Model Checks for Regression: An Innovation Process Approach. Annals of Statistics, 26, 1916-1934. https://doi.org/10.1214/aos/1024691363
[15]	Stute, W. and Zhu, L.X. (2002) Model Checks for Generalized Linear Models. Scandinavian Journal of Statistics, 29, 535-545. https://doi.org/10.1111/1467-9469.00304
[16]	Koul, H.L. and Ni, P. (2004) Minimum Distance Regression Model Checking. Journal of Statistical Planning and Inference, 119, 109-141. https://doi.org/10.1016/S0378-3758(02)00415-9
[17]	Zhu, L., Zhu, R. and Song, S. (2008) Diagnostic Checking for Multivariate Regression Models. Journal of Multivariate Analysis, 99, 1841-1859. https://doi.org/10.1016/j.jmva.2008.01.022
[18]	Theil, H. (1968) A Simplification of the BLUS Procedure for Analyzing Regression Disturbances. Journal of the American Statistical Association, 63, 242-251. https://doi.org/10.1080/01621459.1968.11009238
[19]	Magnus, J.R. and Sinha, A.K. (2005) On Theil’s Errors. Econometrics Journal, 8, 39-54. https://doi.org/10.1111/j.1368-423X.2005.00150.x
[20]	White, H. and MacDonald, G. (1980) Some Large-Sample tests for Non-Normality in the Linear Regression Model. Journal of the American Statistical Association, 75, 16-28. https://doi.org/10.1080/01621459.1980.10477415
[21]	Cliff, H.J. and Ben, B.W. (1974) On the Testing of Regression Disturbances for Normality. Journal of the American Statistical Association, 69, 330-335. https://doi.org/10.1080/01621459.1974.10482948
[22]	Jurečková, J., Picek, J. and Sen, P.K. (2003) Goodness-of-Fit Test with Nuisance Regression and Scale. Metrika, 58, 235-258. https://doi.org/10.1007/s001840300262
[23]	Sen, P.K., Jurečková, J. and Picek, J. (2003) Goodness-of-Fit Test of Shapiro-Wilk Type with Nuisance Regression and Scale. Austrian Journal of Statist, 32, 163-177.
[24]	Shapiro, S.S. and Wilk, M.B. (1965) An Analysis of Variance Test for Normality. Biomatrika, 52, 591-611. https://doi.org/10.1093/biomet/52.3-4.591
[25]	Jurečková, J. and Picek, J. (2007) Shapiro-Wilk Type Test of Normality under Nuisance Regression and Scale. Computational Statistics & Data Analysis, 51, 5184-5191. https://doi.org/10.1016/j.csda.2006.08.026
[26]	Neumeyer, N., Dette, H. and Nagel, E.-R. (2006) Bootstrap Tests for the Error Distribution in Linear and Nonparametric Regression Models. Australian & New Zealand Journal of Statistics, 48, 129-156. https://doi.org/10.1111/j.1467-842X.2006.00431.x
[27]	Hattab, M.W. and Christensen, R. (2018) Lack of Fit Tests Based on Sums of Ordered Residuals for Linear Models. Australian & New Zealand Journal of Statistics, 60, 230-257. https://doi.org/10.1111/anzs.12231
[28]	Christensen, R. and Sun, S.K. (2010) Alternative Goodness-of-Fit Tests for Linear Models. Journal of the American Statistical Association, 105, 291-301. https://doi.org/10.1198/jasa.2009.tm08697
[29]	Christensen, R. and Lin, Y. (2015) Lack-of-fit Tests Based on Partial Sums of Residuals. Communications in Statistics - Theory and Methods, 44, 2862-2880. https://doi.org/10.1080/03610926.2013.844256
[30]	Ramsey, J.B. (1969) Tests for Specification Errors in Classical Linear Least Squares Regression Analysis. Journal of the Royal Statistical Society: Series B (Methodological), 31, 350-371. https://doi.org/10.1111/j.2517-6161.1969.tb00796.x
[31]	Chinchilli, V.M. (1987) Blus Residuals in Multivariate Linear Models. In: Bozdogan, H. and Gupta, A.K., Eds., Multivariate Statistical Modeling and Data Analysis, Springer, Dordrecht, 61-75. https://doi.org/10.1007/978-94-009-3977-6_4
[32]	Vougas, D.V. (2008) Unit Root Testing Based on BLUS Residuals. Statistics & Probability Letters, 78, 1943-1947. https://doi.org/10.1016/j.spl.2008.01.063
[33]	Shapiro, S.S., Wilk, M.B. and Chen, H.J. (1968) A Comparative Study of Various Tests for Normality. Journal of the American Statistical Association, 63, 1343-1372. https://doi.org/10.1080/01621459.1968.10480932
[34]	De Wet, T. and Venter, J.H. (1972) Asymptotic Distributions of Certain Test Criteria of Normality. South African Statistical Journal, 6, 135-149.
[35]	Del Barrio, E., Couesta-Albertos, J.A., Rodríguez-Rodríguez, J.M. and del Barrio, E. (1999) Tests of Goodness of Fit Based on the L2-Wasserstein Distance. The Annals of Statistics, 27, 1230-1239. https://doi.org/10.1214/aos/1017938923
[36]	Scheffe, H. and Tukey, J.W. (1945) Non-Parametric Estimation. I. Validation of Order Statistics. The Annals of Mathematical Statistics, 16, 187-192. https://doi.org/10.1214/aoms/1177731119
[37]	Zhao, J., Xu, X. and Ding, X. (2009) Some New Goodness-of-Fit Tests Based on Stochastic Sample Quantiles. Communications in Statistics - Simulation and Computation, 38, 571-589. https://doi.org/10.1080/03610910802585832
[38]	Zhao, J. and Xu, X. (2014) Goodness-of-Fit Tests for Location?Cscale Families Based on Random Distance. Journal of Statistical Computation and Simulation, 84, 739-752. https://doi.org/10.1080/00949655.2012.725246
[39]	Gan, F.F. and Koehler, K.J. (1990) Goodness-of-Fit Tests Based on P-P Probability Plots. Technometrics, 32, 289-303. https://doi.org/10.2307/1269106
[40]	Krauczi, É. (2007) A Study of the Quantile Correlation Test for Normality. Test, 18, 156-165. https://doi.org/10.1007/s11749-007-0074-6