Estimation of Nonparametric Multiple Regression Measurement Error Models with Validation Data

In this article, we develop estimation approaches for nonparametric multiple regression measurement error models when both independent validation data on covariables and primary data on the response variable and surrogate covariables are available. An estimator which integrates Fourier series estimation and truncated series approximation methods is derived without any error model structure assumption between the true covariables and surrogate variables. Most importantly, our proposed methodology can be readily extended to the case that only some of covariates are measured with errors with the assistance of validation data. Under mild conditions, we derive the convergence rates of the proposed estimators. The finite-sample properties of the estimators are investigated through simulation studies.


Introduction
We can consider the following nonparametric regression model of a scaler response Y on an explanatory variable X ( ) , where ( ) g ⋅ is assumed to be a smooth, continuous but unknown nonparametric regression function and ε is a noise variable with ( ) It is not uncommon that the explanatory variable X is measured with error and instead only its surrogate variable W can be observed.In this case, one observes inde-pendent replicates ( ) ( ) , W Y rather than ( ) , X Y , where the relationship between i W and i X may or may not be specified.If not, the missing information for the statistical inference will be taken from a sample ( ) , j j W X , 1 N j N n + ≤ ≤ + , of so-call validation data independent of the primary (surrogate) sample.The objective of this manuscript is to estimate the unknown function ( ) g ⋅ via the surrogate data .
A wide number of problems of similar type have attracted considerable attention in research literature over the past two decades (see, [1]- [6]).For instance, a quasi-likelihood method is intensively studied by [7].A regression calibration approach is developed by [8] [9] and [10] [11] propose a method based on simulationextrapolation (SIMEX) estimation.Other related methods include Bayesian approaches (see, [12]), semi-parametric method (see, [13] [14]), empirical likelihood method (see, [15]) and the instrumental variable method (see, [16]).Unfortunately, all these work mostly assume some parametric relationships between covariates and responses.Recently, nonparametric estimators of g have been developed by [17] and [18].[17] develops a kernel-based approach for nonparametric regression function estimation with surrogate data and validation sampling.However, his method is not applicable for model (1) since it assumes that the response but not the covariable is measured with error.[18] proposes a nonparametric estimator which integrates local linear regression and Fourier transformation method when both explanatory and surrogate variable are scalars.Nonetheless, their method cannot be extended to multidimensional problems in which the explanatory variable vectors can consist of variables being measured with or/and without errors.For additional references and relevant topics for nonparametric regression models with measurement errors, ones may consult [19] and the references therein.
In practice, nonparametric estimation of g may not be an easy task since, as explained in Section 2, the relation that identifies g is a Fredholm equation of the first kind, i.e.
, Tg m = which may lead to an ill-posed inverse problem.Ill-posed inverse problem related to nonparametric regression model has received considerable attention recently.[20] [21] consider kernel-based estimators while [22] and [23] develop series or sieve estimators.However, their methods require an instrumental variable, and assume that the explanatory variable X is directly observable without errors.In this article, we propose a nonparametric estimation approach which consists of two major steps.First, we propose estimators of generalized Fourier coefficients of T and m based on surrogate and validation data.Second, we replace the infinite-dimensional operator T by the finite-dimensional approximation to avoid higher-order coefficient estimation, and hence it develops an estimator of g.Furthermore, we extend this method to the case that only some of covariates are measured with errors.Under mild conditions, the consistencies of the resulting estimators are established and the convergence rates are also derived.This article is arranged as follows.In Section 2, we first describe our estimation approach for the case that the covariates are all measured with errors.Extension to the case that only some of covariates are measured with errors will be discussed as well.We derive the convergence rates of our estimators under some regularity conditions in Section 3. Section 4 presents some numerical results from simulation studies.A brief discussion will be given in Section 5. Proofs of the theorems are presented in Appendix.

Methodology
We first describe our estimation approach for the case that the covariates are all measured with errors.In addition to the independent and identically distributed (i.i.d.) primary observations ( ) are also available.We shall suppose that X and W are both d-dimensional random vectors.Without loss of generality, let the supports of X and W both be contained in (otherwise, one can carry out monotone transformations of X and W).In the following we let XW f , X f , W f denote respectively the joint density of ( ) , X W , marginal densities of X and W. Then we have According to Equation (3), g is actually the solution to an integral equation called Fredholm equation of the first kind.Let ( ) ( ) ( ) : Hence, Equation ( 3) is equivalent to the operator equation For the unknown smooth function : g χ →  , we assume that s g ∈  where where c is a positive and finite constant.
( ) , and the derivatives here ⋅ denotes the norm on ( ) χ .An estimator of g can then be obtained by replacing T and m by their series estimators based on surrogate data and validation data, and solving the resultant empirical version of (4).As before, let { } , and , , where k m and kl d represent the generalized Fourier coefficients of m and XW f , respectively.Intuitively, we can obtain the estimators of k m , ( ) m w , kl d and ( ) 1 ˆˆ, and , , respectively, where the integer q is a truncation point which is the main smoothing parameter in the approximating Fourier series.The operator T can then be consistently estimated by Define the subset of s  : .
The estimator of ( ) Let n W and n X , respectively, denote the n q × matrices whose ( )   , then the solution to (5) assumes the following form ( ) ( ) where { } is given by ( ) ( ) Next, we extend the estimator in ( 5) to nonparametric regression measurement error models with multi-covariates, that is where X is measured with error and W being its observed surrogate variable, and Z is measured without error.
Let ( ) = be a random sample from model (7), and ( ) be i.i.d.validation observations.We assume that X and W are supported on , and denote respectively the joint density of ( ) , X W , marginal densities of X and W, all conditioning on Z z = .Similar to (3), for any where , and the operator z T is defined by where ( ) To obtain the estimator of ( ) where K is a kernel function and . We consider the following estimators ( ˆ, , and , . for any ( ) , the estimator of ( ) then the solution to (9) has the following form where { } , 1, , is given by ( ) ( )

Theoretical Properties
In this section, we study the asymptotic properties of the estimators proposed in Section 2. We define n τ ( zn τ ) as a sieve measure of ill-posedness (see, [23]): First, we investigate the large-sample properties of the estimator ĝ .For this purpose, we present the following regular conditions which are mild and can be found in [24]) and [23].
A1. (i) The support of ( ) for some constant 0 1 λ < ≤ ; and (ii) ( ) where In (11), the term arises from the bias of ĝ caused by truncating the series approximation of g.The truncation bias decreases as s increases and g becomes smoother.Therefore, the smoother of g the faster the rate of convergence of ĝ .The terms 1 2 1 2 n N q τ − and 1 2 1 2 n n q τ − are respectively induced by random surrogate sampling errors and random validation sampling errors in the estimates of the generalized Fourier coefficients ˆk g .When X is measured without error, the convergence rate of the sieve estimator of g is ( ) Comparing this rate to that in (11), we note that the bias part is of the same order, however, the standard deviation part blows up from A more precise behaviour of the estimator can be obtained but depends on n τ , as [23] discussed, which can be classified into mildly ill-posed case and severely ill-posed case.In the next corollary, we obtain these rates for the two particular cases.( ) , we have (severely ill-posed case) with 1 2 0 r s − − > , and ( ) , we have where the function ( ) L q goes to ∞ slowly such that ( ) 0 L q q ε → for all 0 ε > . .For this purpose, we make the following assumptions.
B1. (i) The support of ( ) = , the joint probability measure of ( ) , Y W is absolutely continuous with respect to the product probability measure of Y and W and; (iii) Conditioning on Z z = , the support of W is a cartesian product of compact con- nected intervals on which W has a probability density function that is bounded away from zero.
B6. (i) lim n N λ = for some constant 0 1 λ < ≤ ; and (ii) ( ) The proofs of all the theorems are reported in Appendix.

Numerical Properties
In this subsection, we conducted a simulation study of the finite-sample performance of the proposed estimators.First, we choose the cosine sequence with ( ) , then get our estimators (denoted as ˆ( ) ) following ( 6) and (10).For comparison, we consider [18] method (denoted as ˆD g ), and used the standard Nadaraya-Watson esti- mator with a Epanechnikov kernel to calculate ˆN g based on the primary dataset.It should be pointed out that ˆN g can serve as a gold standard in the simulation study, even though it is practically unachievable due to mea- surement errors.The performance of estimator est g is assessed by using the average integrated squared errors and ε being distributed as ( ) . To perform this simulation, we generate X from a standard normal distribution, that is, ( ) , and assume that ( ) , and η is the standard deviation of the measurement error.Then, trim X and W in [ ] respectively.For each case, 1000 simulated data sets were generated for each sample size of ( ) It is interesting to compare our estimator ĝ with the estimators ˆD g and ˆN g .Here, since our estimator ĝ involves the regulation parameter q, we therefore present the following cross-validation (CV) selection criterion where the subscript i − meant that the estimator was using the ith observation ( ) g , [18] proposed an automatic way of choosing the smoothing parameters N h , n b and q.For ˆN g , the CV approach is used for choosing bandwidth N h .
Figure 1 shows the regression function curve ( ) g x , and the curves of the median MISEs based on 1000 replicated estimates of ĝ , ˆD g and ˆN g with 0.7 η = under different sample size.From Figure 1, both ĝ and ˆD g successfully capture the patterns of the true regression curves and have smaller bias than ˆN g .As expected, ˆN g fails to produce accurate function curve estimates.In addition, it is obvious that the quality of our proposed estimator improve with the increase of sample sizes.
Table 1 compares, for various sample sizes, the results obtained for estimating curve ( )  Table 1.The estimated MISE ( ( ) estimator generally performs better than the estimator proposed by [18] for the resultant MISEs of ĝ are usually smaller.Also, the performance of ĝ improves (i.e. the corresponding MISEs decrease) considerably as the sample sizes increases.For any nonparametric method in measurement error regression problem, the quality of the estimator also depends on the discrepancy of the observed sample.That is, the performance of the estimator depends on the variances of measurement error.Here, we compare the results for different values of η .
As expected, Table 1 shows that the effect of the variances on the estimator performance is obvious.
Example 2: We considered model (7) with the regression function being and ε being distributed as ( ) was generated from a bivariate normal distribution ( ) and the correlation coefficient between X and Z being 0.6, and ) Here, we only compared our estimator ( ) with the naive estimator N g  which is the multivariate ker- nel regression estimator based on the primary dataset ( ) [18] method cannot be applied to multivariate cases.Here, we used the Epanechnikov kernel function x ≤ for ( ) , g x z  and used an product kernel ( ) ( ) ( ) For the naive estimator N g  , bandwidth selection rules were considered by [25].For our estimator ( ) , we used the cross-validation approach to choosing the three parameters N h , n h and q.For this purpose, n h and ( ) , N h q are selected separately as follows.Define .
Here, we adopt the cross-validation (CV) approach to estimate n h by where the subscript j − denotes the estimator being constructed without using the jth observation.After obtaining ˆn h , we then select ( ) where the subscript i − denotes the estimator being constructed without using the ith observation ( ) We compute MISE at 101 101 × grid points of ( ) for various sample sizes.Table 2 shows that our proposed estimator substantially outperformed the naive kernel estimator N g  .It is obvious that our proposed esti- mator g  has much smaller MISE than N g  .

Discussion
In this paper, we propose a new method for estimating non-parametric regression measurement error models using surrogate data and validation sampling.The covariates are measured with errors while we do not assume any error model structure between the true covariates and the surrogate variable.Most importantly, our proposed method can be readily extended to the multi-covariates model, say, , and .
, , . By the definition of ns  , we have n ns g ∈  .Lemma 1.Under conditions A1 and A3(i) and the sieve space ns  , we have 1) ŝup .

Proof of Theorem 2
Lemma 3.For each , and .Combining all these results, we complete the proof.

Remark 3 .
If Z is discretely distributed with finite support, then ( ) , g x z can be estimated by (9) with ( ) h K u being replaced by ( ) 0 I u = , where ( ) I ⋅ is the indicator function.

2 L
The joint probability measure of ( ) , Y W is absolutely continuous with respect to the product probability measure of Y and W and; (iii) The support of W is a cartesian product of compact connected intervals on which W has a probability density function that is bounded away from zero.A2.For each w χ ∈ , the function ( ) χ , and bounded un- iformly over k.A5.(i) lim n N λ =

Corollary 1 .
Suppose the assumptions of Theorem 1 are satisfied.

Remark 4 . 1 d 5 N
According to Corollary 1(i), the convergence rate becomes ( ) = .This is slower than that of the sieve estimator of a conditional mean function which can achieve the rate of convergence2 − .Next, we study the large-sample properties of the estimator ( ) , g x z 

2 L
χ , and bounded un- iformly over k and; (ii) The kernel function K is a symmetrical, twice continuously differentiable function on

.
The estimated MISEs which were evaluated on a grid of 201 equidistant values of x in [ ] 0,1 are presented.Our results show that the estimators ĝ and ˆD g outperform ˆN g .It is noteworthy that our proposed

.
Then, trim X, W and Z in [ ] reported.Simulations were run with different validation and primary data sizes ( ) each case, 1000 simulated data sets were generated for each sample size of ( ), n N .
z ε = + where x is measured with error but z is measured exactly.Numerical results show that the new estimators are promising in terms of cor-χ χ → denotes the adjoint operator of T. Under assumption A1(ii), the self-adjoint operators of * TT and * T T have the same eigenvalue sequence { }


By some modifications of the proof of Theorem 2 in[23] and applying the Theorem 7 in[24], the proofs of Lemma 1 and Lemma 2 are straightforward and are omitted.Proof of Theorem 1.By the triangle inequality, we have ˆˆ.By the definition of ns  and condition A3(i), we have ( ) .
g.[26] for Fourier series.Next, by the definition of n τ and the triangle inequality, we have( )By conditions A2, A4 and central limit theorem, we can show that By assumption B3(i), it is easy to show that ( )According to assumptions B2, B3(ii), B4, and B5(i), we can show that addition, by some modifications of the proof of Lemma 2, under assumptions B1, B3(ii), B4, B5(i) and B6, we have B1, B3(i) and the sieve space ns  , we have ( ) const.