Estimation of Finite Population Totals in High Dimensional Spaces

Festus A. Were, George O. Orwa, Romanus O. Otieno

Department of Statistics and Actuarial Sciences, JKUAT, Nairobi, Kenya.

DOI: 10.4236/ojs.2022.125035                

Abstract

In this paper, the problem of Nonparametric Estimation of Finite Population Totals in high dimensional datasets is considered. A robust estimator of the Finite Population Total based on Feedforward Backpropagation Neural Network is derived with the aid of a Super-Population Model. This current study is motivated by the fact that Local Polynomials and Kernel methods have in preceding related studies, been shown to provide good estimators for Finite Population Totals but in low dimensions. Even in these situations however, bias at boundary points presents a big challenge when using these estimators in estimating Finite Population parameters. The challenge worsens as the dimension of regressors increase. This is because as the dimension of the Regressor Vectors grows, the Sparseness of the Regressors’ values in the design space becomes unfeasible, resulting in a decrease in the fastest achievable rates of convergence of the Regression Function Estimators towards the target curve, rendering Kernel Methods and Local Polynomials ineffective to address these challenges. This study considers the technique of Artificial Neural Networks which yields robust estimators in high dimensions and reduces the estimation bias with marginal increase in variance. This is due to its Multi-Layer Structure, which can approximate a wide range of functions to any required level of precision. The estimator’s properties are developed, and a comparison with existing estimators was conducted to evaluate the estimator’s performance using real data sets acquired from the United Nations Development Programme 2020. The estimation approach performs well in an example using data from a United Nations Development Programme 2020 on the study of Human Development Index against other factors. The theoretical and practical results imply that the Neural Network estimator is highly recommended for survey sampling estimation of the finite population total.

Keywords

Neural Networks, Kernel Smoother, Local Polynomial, Nonparametric

1. Introduction

In Surveys, extrapolation reduces the accuracy of information since the sample is a subset of an entire population and therefore, does not contain information on units that are not represented in the selected sample. In such cases of unobserved units therefore, use of Auxiliary Information on the characteristic under study is usually effective in predicting unobserved units if the model is correctly specified. In general, when using Auxiliary Information, it is assumed that there is a finite population of N distinct and identifiable units; $U=\left\{1,2,\cdots ,N\right\}$. Let each population unit have the variable of interest as Y. It is assumed that there is an auxiliary variable $X\in {ℝ}^d$, closely correlated with Y, which is known for the entire population (i.e. $X_1\mathrm{,}{X}_2,\cdots \mathrm{,}{X}_N$ ) that is known as $\forall Y_i$. Researchers are frequently faced with the task of estimating a population function, (i.e. a function of Y’s), such as the Population Total;

$T={\displaystyle \underset{i=1}{\overset{N}{\sum }}}{Y}_i$ (1)

or the population distribution functions

$F\left(y\right)=\frac{1}{N}{\displaystyle \underset{i=1}{\overset{N}{\sum }}}{I}_i\left(Y_i\le y\right)$ (2)

In estimating the Population Totals T for instance, a sample S is usually chosen such that the pair $\left(x_{i,j},y_i\right),i=1,2,\cdots ,n$ and $j=1,2,3,\cdots ,d$ is obtained from the variable X and corresponding variable Y. It can then be employed in the design, estimation, or both stages. In the presence of such Auxiliary Variables, Super-Population Models at the estimation stage of inference may be used, [1] and [2]. However, regarding the underlying relationship between the Survey and Auxiliary Variables, all of these techniques refer to Simple Statistical Models (Linear Regression Models). In an Empirical Study, [3] show that misspecification of the model can lead to substantial mistakes in the Parametric Superpopulation. To solve this problem, Nonparametric Regression involving robust estimators in Finite Population Sampling has been proposed [4] [5] [6].

As a result, the reason for using a nonparametric approach in this research is that a regression curve created this way serves four key functions, as explained by [7]: It provides a versatile method of exploring the general relationship between two variables, enables one to make prediction of observations without any reference to fixed parametric model, is a tool for finding spurious observations by studying influence of isolated points and is a flexible method for interpolating between adjacent values of auxiliary variable.

Usually, a major problem that is encountered when using Nonparametric Kernel based Regression Estimators over a finite interval such as the estimation of finite population quantities is the bias at the boundary points, ( [8]). It is also known that Kernel and Polynomial Regression Estimators provide good estimates for the population totals when $x\in {ℝ}^d$ and $d=1$, [5] [9].

Despite the fact that High Dimensional Auxiliary Information can be accounted for in the above estimators, the problem of Regressor Sparseness in the design space renders Kernel Methods and Local Polynomials unworkable because performance decreases quickly as the dimension increases, [9] [10] [11]. This problem is known as “curse of dimensionality” which is a result of the sparsity of data in high-dimensional environments, which leads to a drop in the highest feasible rates of convergence of regression function estimators towards their target curve as the dimension of the Regressor Vector grows. A review on the concept of curse of dimensionality is provided in [12].

Given the problem called “curse of dimensionality”, one has to use different Nonparametric Estimators to retain a large degree of flexibility. An attempt to navigate through this curse while handling Multiple Auxiliary Information is to consider and use recursive covering in model based perspectives [13] and Generalized Additive Modeling in Model-Assisted Framework [14]. These estimation methods come at a cost of reduced flexibility with the associated risk of increased bias [10] [11] [12] [15].

Consequently in this paper, robustness of the proposed Nonparametric Estimator for the Finite Population Total is based on Feedforward Backpropagation Neural Network Approach to address the shortcomings of previously studied estimation methods is developed. Although Kernel and Local Approximators may also have the same property as Artificial Neural Networks (ANNs), they often require a high number of components to attain equivalent approximation accuracy [16]. The high number of components presents a challenge to feasibility in usage of the methods. ANNs are thus considered to be a parsimonious approach to this Parametric Functional Analysis.

2. Estimation of Finite Population Totals Using Artificial Neural Networks

Let Y be the Survey Variable associated with an Auxiliary Variable X assumed to follow a Superpopulation Model under a Model-Based Approach. A commonly used working model for the Finite Population is

$y_i=m\left(x_i\right)+{\epsilon }_i$ (3)

with $x_{ij}\in {ℝ}^d$, ${\epsilon }_1\mathrm{,}{\epsilon }_2\mathrm{,}\cdots \mathrm{,}{\epsilon }_N$ i.i.d with mean zero and $x_{ij},i=1,2,\cdots ,N;j=1,2,\cdots ,d$ are the auxiliary information.

Also, let

$T={\displaystyle \underset{i\in s}{\sum }}{y}_i+{\displaystyle \underset{i\in r}{\sum }}{y}_i$ (4)

be the finite population total where s is the sampled units and r are the non-sampled units. Assume that $y_i$ is given according to Equation (3) with $x_i\in {ℝ}^d$, ${\epsilon }_1\mathrm{,}{\epsilon }_2\mathrm{,}\cdots \mathrm{,}{\epsilon }_N$ i.i.d. Consider estimating $m\left(x\right)$ based on a Feedforward Backpropagation Neural Network. As a basic building block, consider the Neurons as a Nonlinear Transformation of a Linear Combination of the input $x={\left(x_1\mathrm{,}\cdots \mathrm{,}{x}_d\right)}^{\prime }$.

Feedforward networks with multiple layers of hidden units are more complex networks that enable information feedback to be specified. Its study will only deal with the presented structure 5, which is widely used for a range of applications and has the appealing characteristic of being implemented in statistical software, but the results herein are straightforward to extrapolate.

In this simplest case of one hidden layer with $H\ge 1$ Neurons, the Network can be written to represent the Network Function as follows

$f_H\left(x,\theta \right)=v_0+{\displaystyle \underset{h=1}{\overset{H}{\sum }}}{v}_h\psi \left(w_{0h}+x^{\text{T}}{w}_h\right),x\in {ℝ}^d$ (5)

with $w_h=\left(w_{1h},\cdots ,w_{dh}\right)\in {ℝ}^d$ and

$\theta ={\left(w_{01},\cdots ,w_{0H},w_1^{\text{T}},\cdots ,w_H^{\text{T}},v_0,\cdots ,v_h\right)}^{\text{T}}\in {ℝ}^{M\left(H\right)}$ (6)

where $M\left(H\right)=\left(d+1\right)H+H+1$ represents the vector of all parameters of weights of the network. $\psi \mathrm{<mo>\; :\; </mo>}ℝ\mapsto ℝ$ is a given Activation Function. For regression problems, functions of the sigmoid shape. Therefore, depending on the required output, one could choose between widely used sigmoid functions, the logistic sigmoid and the bipolar sigmoid. The Logistic Function is preferable when the objective is to approximate functions that map into probability space. In particular, the Activation Function is a smooth counterpart of the Indicator Function if the input signals are “constrained” between zero and one. For instance, logistic function described as

$\psi \left(u\right)=\frac{1}{1+\exp \left(-u\right)},-\infty <u<\infty$ (7)

is a leading example of which it approaches one (zero) when its arguments go to infinity (negative infinity). Thus, the Logistic Activation Function produces partially on/off signals following the received input signals. This function $f_H\left(x\mathrm{<mo>\; ;\; </mo>}\theta \right)$ specifies a mapping from the input space ${ℝ}^d$ to the output space which for this study is one-dimensional. Such a class of all network output function $O=\left\{f_H\left(x;\theta \right),\theta \in {ℝ}^{M\left(H\right)},H\ge 1\right\}$ has several uniform approximation properties [17] [18] [19]. Important for the current study is that for any continuous function m, any $\epsilon >0$ and any compact set $C\subseteq {ℝ}^d$ there exist a function $f_H\in O$ with

$\underset{x\in C}{\sup }\left|m\left(x\right)-f_H\left(x;\theta \right)\right|<\epsilon$

These imply that any Regression Function $m\left(x\right)$ may be approximated well enough using a large enough number of neurons and appropriate parameters $\theta$.

Therefore, a nonparametric estimate for $m\left(x\right)$ is gotten if H is first chosen in a manner which serves as a tuning parameter and determines the smoothness of the estimate, then estimation of the parameter $\theta$ from the data by nonlinear least squares is done to yield

${\hat{\theta }}_n=\arg \underset{\theta \in {\Re }^{M\left(H\right)}}{\min }{D}_n\left(\theta \right)$ (8)

with

$D_n\left(\theta \right)={\displaystyle \underset{s}{\sum }}{\left(y_i-f_H\left(x;\theta \right)\right)}^2$

Under appropriate conditions, ${\hat{\theta }}_n$ converges in probability for $n\to \infty$ and a constant H to the parameter vector $\theta \in {\Theta }_H$ which corresponds to the best approximation of $m\left(x\right)$ by a function of type $f_H\left(x\mathrm{<mo>\; ;\; </mo>}\theta \right)\mathrm{,}\theta \in {\Theta }_H$ with

$\theta =\arg \underset{\theta \in {\Re }^{M\left(H\right)}}{\min }D\left(\theta \right)\text{with}D\left(\theta \right)=E\left\{m\left(x\right)-f_H\left(x;\theta \right)\right\}$

Also, under some stronger assumptions, the Asymptotic Normality of ${\hat{\theta }}_n$ and thus the estimator of $\hat{m}\left(x\right)=f_H\left(x\mathrm{<mo>\; ;\; </mo>}{\hat{\theta }}_n\right)$ also follows for the regression function $m\left(x\right)$. Therefore, the immediate consequence of these is that $f_H\left(x\mathrm{<mo>\; ;\; </mo>}{\hat{\theta }}_n\right)\to f_H\left(x\mathrm{<mo>\; ;\; </mo>}\theta \right)$ as $n\to \infty$.

The estimation error ${\hat{\theta }}_n-\theta$ can be divided into two asymptotically independent subcomponents: ${\hat{\theta }}_n-\theta =\left({\hat{\theta }}_n-{\hat{\theta }}_n\right)+\left({\hat{\theta }}_n-\theta \right)$, where the value

${\theta }_n=\arg \underset{\theta \in {\Re }^{M\left(H\right)}}{\min }{\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\left\{m\left(x\right)-f_H\left(x,\theta \right)\right\}}^2$

minimises the sample version of $D\left(\theta \right)$, [20]. Thus, by Universal Approximation Property of Neural Networks, $f_H\left(x\mathrm{<mo>\; ;\; </mo>}\theta \right)$ converges to the Regression Function $m\left(x\right)$ as $H\to \infty$. Therefore $f_H\left(x\mathrm{<mo>\; ;\; </mo>}{\hat{\theta }}_n\right)$ is a consistent Estimate of $m\left(x\right)$ if H increases with n as is herein imposed, and with an appropriate rate. From these results, the corresponding estimate of the finite population total is therefore, given as

${\hat{T}}_{NN}={\displaystyle \underset{j\in s}{\sum }}{y}_j+{\displaystyle \underset{j\in r}{\sum }}{\hat{m}}_n\left(x_j\right)$ (9)

which is the proposed estimator for the Finite Population Total, with

${\hat{m}}_n\left(x_j\right)=f_H\left(x;{\hat{\theta }}_n\right)$

Regularity Notes on the Proposed Estimator

1) $T_{NN}$ is a Model-Based Estimator, so that all the inference is with respect to the model for the ${y^{\prime }}_{i}s$, not the Survey Design.

2) This estimator is identical to that proposed in [4], except that the NN is replaced by a Kernel-Based Regression.

3) This estimator can be used to estimate the population totals of a finite population so long as the assumption is that each of the unsampled elements has the same distribution as the sampled elements.

4) For fixed H, this work just fits a Nonlinear Regression Model to the data. However, it is known that this model can be misspecified and therefore one has to select a decent H, determining the form of the nonlinear regression function and the dimension of its parameter, to get a reasonable balance between bias and variance of ${\hat{m}}_n\left(x\right)$ as an estimate of $m\left(x\right)$.

5) The parameter vector $\theta$ of [5] is not uniquely determined (identified) by the function $f_H\left(x\mathrm{,}\theta \right)$. i.e. for different values of $\theta$, the same function $f_H\left(x\mathrm{,}\theta \right)$ is realised. If, for example the activation function is antisymmetric, $\psi \left(-x\right)=-\psi \left(x\right)$, then changing the enumeration of hidden units and multiplying all weights $w_{ih},i=1,2,\cdots ,d$, going into hidden units and simultaneously the weight $v_h$ going out of the neuron by −1 do not change the function. To avoid this ambiguity and the related problems of estimation, this study considered only parameter vectors in a subset ${\Theta }_H\subset {ℝ}^{M\left(H\right)}$ chosen such that for each function in [5] with H neurons, there exists exactly one corresponding parameter ${\Theta }_H$. For antisymmetric $\psi$ one can choose for example ${\Theta }_H=\left\{\theta \in {ℝ}^{M\left(H\right)}\mathrm{<mo>\; ;\; </mo>}{v}_1\ge v_2\ge \cdots \ge v_H\right\}$, that is, the last h coordinates of $\theta$ are in decreasing order. For more details on the identification of parameters see [21].

Theoretically, Feedforward Neural Network which has one hidden layer suffices by the Universal Approximation Property. For practical purposes, networks with more than one hidden layer may provide a better approximation to $m\left(x\right)$ with fewer parameters, see [9] [17] [18] [22] [23].

3. Theoretical Properties of the Proposed Estimator

3.1. Assumptions

To be able to prove the theoretical results, the following assumptions are made;

1) The errors ${\epsilon }_i$ are Identically Independently Distributed (IID) with mean 0, finite variance ${\sigma }^2$ satisfying

$pr\left(\left|{\epsilon }_i\right|>t\right)\le a_0\exp \left\{-a_1{t}^{\alpha }\right\}\text{for}\text{all}t\ge 0$

and for some $a_0,a_1$ and $\alpha >0$.

2) The Auxiliary Measurements $x_i\in {ℝ}^d$ are i.i.d. with an absolutely continuous distribution F having a finite second moment.

${\displaystyle {\int }_{-\infty }^{x_1}}{\displaystyle {\int }_{-\infty }^{x_2}}\cdots {\displaystyle {\int }_{-\infty }^{x_d}}f\left(t_1,\cdots ,t_d\right)\text{d}t$ (10)

where $f\left(\mathrm{.}\right)$ is strictly positive density whose support is a compact subset of ${ℝ}^d$. Moreover,

$pr\left(\Vert x_i\Vert >t\right)\le b_0\exp \left\{-b_1{t}^{\beta }\right\}\text{for}\text{all}t\ge 0$ (11)

and for some $b_0,b_1$ and $\beta >0$.

3) $m\left(x\right)$ is a bounded function.

4) For each sequence of finite population indexed by v, conditioned on the value $x_i$, the super population model (3.1), where ${\epsilon }_i$ satisfies A1, then, the $x_i$ is considered fixed with respect to the super population model $\xi$.

5) The survey variable has a bounded moment with ξ-probability 1. Moreover, it is noted that (A1), …, (A3) immediately imply for some $c_0,c_1>0$

$Pr\left(\left|y_i\right|>t\right)\le c_0\exp \left\{-c_1{t}^{\alpha }\right\}\mathrm{,}\text{for}\text{all}t\ge 0$ (12)

6) The sampling rate is bounded, that is

$\underset{v\to \infty }{\lim }\sup \frac{n}{N}=\pi ,\text{where}\pi \in \left(0,1\right)$

7) The parameter space $\Theta$ is a compact set, $\theta$ an interior point of $\Theta$ and it is irreducible; that is for $h\mathrm{,}{h}^{\prime }\ne 0$ none of the following three cases holds [21].

a) $v_h=0$, for some $h=1,\cdots ,H$.

b) $w_h=0$, for some $h=1,\cdots ,H$.

c) $\left({w^{\prime }}_h\mathrm{,}{w}_{0h}\right)=\pm \left({w^{\prime }}_{h^{\prime }}\mathrm{,}{w}_{0h^{\prime }}\right)$, for $w\ne w^{\prime }$.

8) The activation function $\psi$ in 7 is asymmetric sigmoid function that is differentiable to any order. Additionally, it is assumed that the class of functions $\left\{\psi \left(b_t,b_0\right),b>0\right\}\cup \left\{\psi \equiv 1\right\}$ is linearly independent. Such function can easily be represented using an indicator (threshold) function,

$\psi \left(u\right)=(\begin{array}{l}\psi \left(u\right)\to 0,\text{as}u\to -\infty \\ \psi \left(u\right)\to 1,\text{as}u\to +\infty \\ \psi \left(u\right)+\psi \left(-u\right)=1\end{array}$ (13)

The logistic activation function in 7 fulfills these requirements.

To prove for consistency of the proposed estimator, the rate which determines how the complexity of the networks and therefore the possible roughness of the function estimate ${\hat{m}}_n\left(x\right)$ increases with the sample size n has to satisfy some conditions. We follow [19] and restrict the number H of neurons and the overall size of the network weights $v_h\mathrm{,}{w}_{kh}$ simultaneously. For some sequences $H_n\mathrm{,}{\Delta }_n\to \infty$, let

${\Theta }_n=\Theta \left(H_n,{\Delta }_n\right)=\left\{\theta \in \Theta ;{\displaystyle \underset{h=0}{\overset{H_n}{\sum }}}\left|{\upsilon }_h\right|\le {\Delta }_n,{\displaystyle \underset{h=1}{\overset{H_n}{\sum }}}{\displaystyle \underset{k=0}{\overset{d}{\sum }}}\left|{\omega }_{kd}\right|\le H_n{\Delta }_n\right\}$ (14)

For given sample size n, we consider only network functions in

$O_n=O\left(H_n,{\Delta }_n\right)=\left\{f_{H_n}\left(x,\theta \right);\theta \in \Theta \left(H_n,{\Delta }_n\right)\right\}$ (15)

as an estimate for $m\left(x\right)$. Therefore, we redefine the parameter estimate as

${\hat{\theta }}_n=\arg \underset{\theta \in {\Theta }_n}{\min }{\displaystyle \underset{s}{\sum }}{\left(y_i-f_H\left(x;\theta \right)\right)}^2$ (16)

and the network estimate for $m\left(x\right)$ is therefore given by

${\hat{m}}_n\left(x\right)=f_{H_n}\left(x,{\hat{\theta }}_n\right)$ (17)

which is a kind of sieve estimate in the sense of [24] or [25].

To prove consistency of ${\hat{T}}_{NN}$, it needs to be shown that the Neural Network Based Regression Function ${\hat{m}}_{NN}$ is also consistent.

Theorem 3.1. Let $\left(y_1\mathrm{,}{x}_1\right)\mathrm{,}\cdots \mathrm{,}\left(y_n\mathrm{,}{x}_n\right)$ be i.i.d variable with $y_i\in ℝ$, and $x_i\in {ℝ}^d$ . Let the distributions of $y_i$ and $x_{x_i}$ satisfy A2 and Equation (12). Let $O_n=O\left(H_n,{\Delta }_n\right),n\ge 1$ be the set of neural network output functions given by Equation (15) with an activation function $\psi$ which is Lipschitz continuous on $ℝ$, strictly increasing and satisfying Equation (13). Let ${\hat{m}}_n\left(x\right)=E\left(y_i|x_i\right)=x$ be in the closure of ${\displaystyle {\cup }_{n=1}^{\infty }}{O}_n$ in $L^2\left(F\right)$ that is, in the space of functions square integrable with respect to the distribution of the $x_i$ . Then ${\hat{m}}_n\left(x\right)$ is a consistent estimate of $m\left(x\right)$ in the $L^2\left(F\right)$ -sense, that is

${\displaystyle \int }{\left(m\left(x\right)-{\hat{m}}_n\left(x\right)\right)}^2\text{d}F\left(x\right)\to 0$ in probability (18)

provided that $H_n\mathrm{,}{\Delta }_n\to \infty$ such that

${\Delta }_n=o\left(n^{\frac{1}{4}}\right)$

$H_n,{\Delta }_n^4\log n=o\left(n\right)$ and $H_n\log n=o\left({\Delta }_n^{\alpha }\right)$

where $\alpha$ determine the rate of decrease of the tail of the distribution of the $y_i$ by Equation (12).

Proof. Theorem 1 can be proven exactly as Theorem 2.1 of [26] for stationary processes satisfying an $\alpha$ -mixing condition and also as Theorem 3.1 of [27] for fixed data. As here the data are independent, the Bernstein inequality for stationary processes may be replaced by a Bernstein inequality for independent data like that one in Section (2.5.4), Lemma A of [28] [29]. Therefore, the right hand side of Equation (5.1) of [26] changes to

$c_1\exp \left(-c_2\frac{\Delta }{N{M}_N^2}\right)\text{instead}\text{of}{c}_1\exp \left(-c_2\frac{{\Delta }^2}{\sqrt{N}{M}_N^2}\right)$

Then the proof proceeds exactly as in [19] and results in slightly different condition for the rates of $H_n\mathrm{,}{\Delta }_n$ in the independence case.

We remark that for bounded random variables $\left(y_i\mathrm{,}{x}_i\right)$, the last condition on $H_n\mathrm{,}{\Delta }_n$ involving $\alpha$ can be dropped. In that case, Theorem 1 essentially is equivalent to Theorem 3.3 of [19]. We also remark that by Theorem 3.4 of [19], we may determine the parameters $H_n\mathrm{,}{\Delta }_n$ which determine the network complexity and therefore the smoothness of the function estimate, adaptively from the data by Cross Validation without changing the consistency of ${\hat{m}}_n\left(x\right)$. For the detail on the proof of these theorems, see the work of [26] [27].

Note that, to prove the consistency of ${\hat{T}}_{NN}$ we need Equation (13) with a simple mean over the unobserved $x_i\mathrm{,}i\in r$ instead of the integral. The following results show that the difference between the integral and the sample mean is negligible.

Theorem 3.2. Let $\left(\left(y_1\mathrm{,}{x}_1\right)\mathrm{,}\cdots \mathrm{,}\left(y_N\mathrm{,}{x}_N\right)\right)$ be i.i.d with 3 for some bounded $m\left(x\right)$ . Let F denote the distribution of $x_i$ . Let $\left|\psi \left(u\right)\right|\le 1$ . Let $s=1,\cdots ,n$ be the index set of the observed data and $r=n+1,\cdots ,N$ the index of unobserved data. Let ${\hat{\theta }}_n$ be defined as in Equation (13) with ${\hat{m}}_n\left(x\right)$ defined as in Equation (17) with ${\hat{m}}_n\left(x\right)=f_{H_n}\left(x,{\hat{\theta }}_n\right)$ denote the estimate of $m\left(x\right)$ based on the sample $\left(y_i\mathrm{,}{x}_i\right)\mathrm{,}i\in s$ . Let $n\mathrm{,}N\to \infty$ such that $\frac{n}{N}\to \pi \left(\mathrm{0,1}\right)$ and let $H_n\mathrm{,}{\Delta }_n$ satisfy conditions in Theorem 3. Then for $\delta >0$

$\begin{array}{l}Pr(|\frac{1}{N-n}{\displaystyle \underset{j\in r}{\sum }}{\left(m\left(x_j\right)-{\hat{m}}_n\left(x_j\right)\right)}^2\\ \begin{array}{c}\\ \end{array}-{\displaystyle \int }{\left(m\left(x_j\right)-{\hat{m}}_n\left(x_j\right)\right)}^2\text{d}F\left(x\right)|>\delta \mathrm{<mo>\; |\; </mo>}\left(y_i\mathrm{,}{x}_i\right)\mathrm{,}i\in s)\le d_1\exp \left\{-d_2\frac{N{\delta }^2}{{\Delta }_n^4}\right\}\end{array}$ (19)

for all $\delta >0$ and all N large enough where $d_1\mathrm{,}{d}_2$ are some constants independent of $N\mathrm{,}n$ and $\left(y_i\mathrm{,}{x}_i\right)\mathrm{,}i\in s$.

Proof. From assumption A3, let C be the upper bound of $m\left(x\right)$. By definition of ${\hat{m}}_n\left(x\right)=f_{H_n}\left(x,{\hat{\theta }}_n\right)$ and $O\left(H_n\mathrm{,}{\Delta }_n\right)$, we immediately have

$\left|{\hat{m}}_n\left(x\right)\right|\le {\Delta }_n\text{a}\text{.s}\left|\psi \left(u\right)\right|\le 1$

setting

$V_{N_i}={\left(m\left(x_j\right)-{\hat{m}}_n\left(x_j\right)\right)}^2-{\displaystyle \int }{\left(m\left(x_j\right)-{\hat{m}}_n\left(x_j\right)\right)}^2\text{d}F\left(x\right),i\to r$ (20)

these therefore result to

$\begin{array}{l}\left|V_{N_i}\right|\le 4\left(C^2+{\Delta }_n^2\right)\\ E\left\{V_{N_i}|\left(y_i,x_i\right),i\in s\right\}=0\\ E\left\{V_{N_i}^2|\left(y_i,x_i\right),i\in s\right\}\le 32\left(C^4+{\Delta }_n^4\right)\end{array}$ (21)

note that ${\hat{m}}_n\left(x\right)$ is independent of $\left(y_i\mathrm{,}{x}_i\right)\mathrm{,}i\in r$, and completely determined by $\left(y_i\mathrm{,}{x}_i\right)\mathrm{,}i\in s$. Now apply Bernstein’s inequality (Lemma A, Section 2.5.4) of [28] and get

$\begin{array}{l}Pr\left(\frac{1}{N-n}\left|{\displaystyle \underset{j\in r}{\sum }}{V}_{N_j}\right|>\delta \mathrm{<mo>\; |\; </mo>}\left(y_i\mathrm{,}{x}_i\right)\mathrm{,}i\in s\right)\\ \le 2\exp \left\{-\frac{N_n{\delta }^2}{64\left(C^4+{\Delta }_n^4\right)+\frac{2}{3}4\left(C^2+{\Delta }_n^2\right)\delta }\right\}\end{array}$ (22)

Now the results follow as ${\Delta }_n\to \infty$ and therefore ${\Delta }_n^4$ dominates the denominator of the exponent for N large enough and as $N-n$ coincides asymptotically with $\left(1-\pi \right)N$. Moreover, as ${\Delta }_n=o\left(n^{\frac{1}{4}}\right),\frac{N}{{\Delta }_n^4}\to \infty$, that is, the right hand side of the inequality converges to zero (taking limits as ${\Delta }_n\to \infty$ ).

3.2. Asymptotic Consistency

Theorem 3.3. If (A1)-(A8) are satisfied and if the activation function $\psi \left(u\right)$ is Lipschits continuous and strictly increasing and also Theorem 1 holds, then the neural network estimate ${\hat{T}}_{NN}$ of the population total T given by 6 with ${\hat{m}}_n\left(x\right)=f\left(x\mathrm{,}{\hat{\theta }}_n\right)$ and ${\hat{\theta }}_n$ given by [8] is consistent in the following sense.

$\begin{array}{l}\frac{1}{N}\left|T-{\hat{T}}_{NN}\right|\to 0inprobability\\ whereN\mathrm{,}n\to \infty with\frac{n}{N}\to \pi \in \left(\mathrm{0,1}\right)\end{array}$ (23)

provided that the number $H_n$ and the bound ${\Delta }_n$ of the network weights satisfy $H_n\mathrm{,}{\Delta }_n\to \infty$ such that

$\begin{array}{l}{\Delta }_n=o\left(n^{\frac{1}{4}}\right)\\ H_n{\Delta }_n^4\log n=o\left(n\right)\\ H_n\log n=o\left({\Delta }_n^{\alpha }\right)\end{array}$ (24)

where $\alpha$ determines(by A1) how fast the tail probability of the ${\epsilon }_i$ and $y_i$ decreases. [19] showed that, the appropriate choice for ${\Delta }_n$ is such that ${\Delta }_n\to \infty$

as $n\to \infty$ and ${\Delta }_n=o\left(n^{\frac{1}{4}}\right)$, i.e. $n^{\frac{1}{4}}{\Delta }_n\to 0$ as $n\to \infty$

Proof.

$\begin{array}{l}\frac{1}{N}\left|T-{\hat{T}}_{NN}\right|=\frac{1}{N}\left|{\displaystyle \underset{j\in r}{\sum }}\left(y_j-{\hat{m}}_n\left(x_j\right)\right)\right|\\ =\frac{1}{N}\left|\left(m\left(x_j\right)-{\hat{m}}_n\left(x_j\right)\right)+{\displaystyle \underset{j\in r}{\sum }}{\epsilon }_j\right|\\ \le \frac{1}{N}\left|m\left(x_j\right)-{\hat{m}}_n\left(x_j\right)\right|+\frac{N-n}{N}\left|\frac{1}{N-n}{\displaystyle \underset{j\in r}{\sum }}{\epsilon }_j\right|\\ \frac{1}{N}{\left(m\left(x_j\right)-{\hat{m}}_n\left(x_j\right)\right)}^2+\frac{N-n}{N}\left|\frac{1}{N-n}{\displaystyle \underset{j\in r}{\sum }}{\epsilon }_j\right|\end{array}$ (25)

by Jensen’s inequality.

Now the last term converges to

$\frac{N-n}{N}\left|\frac{1}{N-n}{\displaystyle \underset{j\in r}{\sum }}{\epsilon }_j\right|=\left(1-\pi \right)\left|E\left({\epsilon }_j\right)\right|$

where $\left(1-\pi \right)\left|E\left({\epsilon }_j\right)\right|=0$ since $E\left({\epsilon }_j\right)=0$ by law of large numbers. The first term of 25 decomposes into

$\begin{array}{l}\frac{N-n}{N}\left(\frac{1}{N-n}{\displaystyle \underset{j\in r}{\sum }}{\left(m\left(x_j\right)-{\hat{m}}_n\left(x_j\right)\right)}^2-{\displaystyle \int }{\left(m\left(x_j\right)-{\hat{m}}_n\left(x_j\right)\right)}^2\text{d}F\left(x\right)\right)\\ +\frac{N-n}{N}{\displaystyle \int }{\left(m\left(x_j\right)-{\hat{m}}_n\left(x_j\right)\right)}^2\text{d}F\left(x\right)\end{array}$ (26)

The right hand terms of 26 converge to 0 by Theorem 1 and as $\frac{N-n}{N}\to 1-\pi$.

The proof is completed by using Theorem 2 to cope with left hand terms where we drop the factor $\frac{N-n}{N}$ converges to $1-\pi$ anyhow.

$\begin{array}{l}Pr\left(\left|\frac{1}{N-n}{\displaystyle \underset{j\in r}{\sum }}{\left(m\left(x_j\right)-{\hat{m}}_n\left(x_j\right)\right)}^2-{\displaystyle \int }{\left(m\left(x_j\right)-{\hat{m}}_n\left(x_j\right)\right)}^2\text{d}F\left(x\right)\right|>\delta \right)\\ =E\left\{Pr\left[\left|\frac{1}{N-n}{\displaystyle \underset{j\in r}{\sum }}{\left(m\left(x_j\right)-{\hat{m}}_n\left(x_j\right)\right)}^2-{\displaystyle \int }{\left(m\left(x_j\right)-{\hat{m}}_n\left(x_j\right)\right)}^2\text{d}F\left(x\right)\right|>\delta \mathrm{<mo>\; |\; </mo>}\left(y_i\mathrm{,}{x}_i\mathrm{,}i\in s\right)\right]\right\}\\ \le d_1\exp \left\{-d_2\frac{N\delta }{{\Delta }_n^4}\right\}\to 0\forall \delta >0\text{as}n\to \infty \mathrm{,}{\Delta }_n\to \infty \end{array}$ (27)

hence the proof.

3.3. Mean Squared Error

Mean Squared Error is used to measure the accuracy of the estimator among other measures of performance. The MSE is defined by $E{\left(T_{NN}-T\right)}^2$ where T denotes the true population total. To estimate $E{\left(T_{NN}-T\right)}^2$, first, we consider

$\begin{array}{l}E\left[{\left(T_{NN}-T\right)}^2\mathrm{<mo>\; |\; </mo>}D\mathrm{,}{X}_{n+1}^N\right]=E\left[{\left({\displaystyle \underset{i=1}{\overset{H}{\sum }}}{\displaystyle \underset{j=n+1}{\overset{N}{\sum }}}\hat{m}\left(x\mathrm{,}\theta \right)-{\displaystyle \underset{j=n+1}{\overset{N}{\sum }}}\left(m\left(x\right)+\epsilon \right)\right)}^2\mathrm{<mo>\; |\; </mo>}D\mathrm{,}{X}_{n+1}^N\right]\\ =\frac{{\left(N-n\right)}^2}{N^2}E\left[{\left(\frac{1}{N-n}{\displaystyle \underset{i=1}{\overset{H}{\sum }}}{\displaystyle \underset{j=n+1}{\overset{N}{\sum }}}\hat{m}\left(x\mathrm{,}\theta \right)-{\displaystyle \underset{j=n+1}{\overset{N}{\sum }}}\left(m\left(x\right)+\epsilon \right)\right)}^2\mathrm{<mo>\; |\; </mo>}D\mathrm{,}{X}_{n+1}^N\right]+\frac{N-n}{N}var\left({\epsilon }_i\right)\\ =\frac{{\left(N-n\right)}^2}{N^2}E\left[{\left(\frac{1}{H\left(N-n\right)}{\displaystyle \underset{i=1}{\overset{H}{\sum }}}{\displaystyle \underset{j=n+1}{\overset{N}{\sum }}}\hat{m}\left(x\mathrm{,}\theta \right)-E\left(T_k\mathrm{<mo>\; |\; </mo>}D\mathrm{,}{X}_j\right)+E\left(T_k\mathrm{<mo>\; |\; </mo>}D\mathrm{,}{X}_j\right)-E\left(T_k\right)\right)}^2\right]+\frac{N-n}{N}var\left({\epsilon }_i\right)\\ =\frac{{\tau }_D^2}{H}\left(1-f\right){\left\{E\left(T_k\mathrm{<mo>\; |\; </mo>}D\mathrm{,}{X}_j\right)-E\left(T_k\right)\right\}}^2+\frac{1-f}{N}var\left({\epsilon }_i\right)\end{array}$ (28)

where the $X_{j=\left(x_{n+1}\mathrm{,}\cdots \mathrm{,}{x}_N\right)}$ is a set of unsampled auxiliary units. $T_k$ denotes the total of the unsampled elements and $E\left(T_k\right)={\displaystyle {\sum }_{j=n+1}^N}m\left(x\right)$.

The last approximation of Equation (28) follows from Equation (15) of [30], that is

$E{\left(\frac{1}{HN}{\displaystyle \underset{i=1}{\overset{H}{\sum }}}{\displaystyle \underset{j=n+1}{\overset{N}{\sum }}}\hat{m}\left(x,\theta \right)-\left(1-f\right)E\left(T_k\right)|D,X_j\right)}^2\approx \frac{{\tau }_D^2}{H}$

for some positive constant ${\tau }_D^2$.

The term $E\left(T_k\mathrm{<mo>\; |\; </mo>}D\mathrm{,}{X}_j\right)-E\left(T_k\right)$ is the predictor bias due to randomness or sampling bias of D. Now from Equation (28), we have

$E{\left(T_{NN}-T\right)}^2=E\left(\frac{{\hat{\tau }}_D^2}{H}\right)+{\left(1-f\right)}^{2}E{\left\{E\left(T_k\mathrm{<mo>\; |\; </mo>}D\mathrm{,}{X}_j-E\left(T_k\right)\right)\right\}}^2+\frac{1-f}{N}var\left({\epsilon }_i\right)$ (29)

As noted in [30], the quantity ${\tau }_D^2$ can be estimated by batch method. Therefore,

${\hat{\tau }}_D^2=\frac{s}{r-1}{\displaystyle \underset{t=1}{\overset{r}{\sum }}}{\left({\hat{T}}_{NN,t}-T_{NN}\right)}^2$ (30)

for details see [30]. Equation (30) can be substituted in 29 in lieu of $E\left({\tau }_D^2\right)$.

Now, under the assumption that the $\frac{{\epsilon }_i}{\sigma }~t\left(v\right)$, then the estimate of $var\left({\epsilon }_i\right)$ is given as

$v\hat{a}r\left({\epsilon }_i\right)=\frac{v}{v-2}\frac{1}{H}{\displaystyle \underset{i=1}{\overset{H}{\sum }}}{\hat{\sigma }}_i^2$ (31)

Under the assumption that the population is made up of exact copies of the sampled (training) data, we have $E\left(T_k\mathrm{<mo>\; |\; </mo>}D\mathrm{,}{X}_j\right)-E\left(T_k\right)\approxeq \hat{T}-T$ where $\hat{T}$ the fitted sample totals and

$E{\left(\hat{T}-T\right)}^2={\left({\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\hat{\epsilon }}_i\right)}^2=Var\left({\hat{\epsilon }}_i\right)$ (32)

Under the true model, we have $Var\left({\hat{\epsilon }}_i\right)=var\left({\epsilon }_i\right)$. Hence the $E{\left\{E\left(T_k\mathrm{<mo>\; |\; </mo>}D\mathrm{,}{X}_j-E\left(T_k\right)\right)\right\}}^2$ can be estimated by

$\hat{B}ia{s}^2=\frac{1}{n}v\hat{a}r\left({\epsilon }_i\right)$ (33)

Thus, $E{\left(T_{NN}-T\right)}^2$ can be estimated by

$\begin{array}{c}\hat{E}{\left(T_{NN}-T\right)}^2=\frac{{\hat{\tau }}_D^2}{H}+\left(1-f\right)B\hat{i}a{s}^2+\frac{1-f}{N}v\hat{a}r\left({\epsilon }_i\right)\\ \mathrm{<mo>\; =\; </mo>}\frac{{\hat{\tau }}_D^2}{H}+\frac{1-f}{n}v\hat{a}r\left({\epsilon }_i\right)\end{array}$ (34)

As $H\to \infty$ Equation (34) reduces to

$\hat{E}{\left(T_{NN}-T\right)}^2=\frac{1-f}{n}v\hat{a}r\left({\epsilon }_i\right)$ (35)

4. Empirical Results

To illustrate our estimation approach, the following data will be utilized. A population of size 188 will be obtained from the United Nations Development Programme 2020 report. The UN studied the development in 1889 countries. It grouped development in the countries as either very high human development, high human development, medium human development or low human development. Kenya was classified in countries that fall under medium development and ranked number 143 among the 188 countries studied. The UN study used attributes such as Human Development Index (HDI), Life expectancy at Birth, Expected years of schooling, Mean years of schooling, Gross national income (GNI) per capita and GNI per capita rank minus HDI to rank human development index in the 189 countries. In this study, a relationship between Human Development Index (HDI) which is considered as the survey variable and the auxiliary variables; Life expectancy at Birth, Expected years of schooling, Mean years of schooling and Gross National Income (GNI) per capita is considered.

In order to understand how the proposed estimator compares against other existing non-parametric regression estimators, we compared the performance of our estimator to that of identified estimators based on Multivariate Additive Regression Splines (MARS), Generalized Additive Models (GAM) and Local polynomial (LP) which can handle high dimensional data. We compare the performance of the proposed estimator of the population totals, with ${\hat{T}}_{LP}$, ${\hat{T}}_{MARS}$, ${\hat{T}}_{GAM}$ and ${\hat{T}}_{SAM}$, using the bias, mean squared error (MSE), mean absolute error (MAE), and mean absolute percentage error (MAPE).

The unconditional results for the estimators were computed that are used in the analysis that acts as performance indicators of the estimators. The results include; Bias, Mean Square Error (MSE), Mean Absolute Error (MAE) and mean absolute percentage error (MAPE) respectively. These criteria are defined as follows; Bias of a Population total estimator refers to the deviation of the expected value of the estimator from the true Total value. Table 1 provides the results for performance of the estimators when applied to the data obtained from the United Nations Development Programme 2020 report. All of the population total estimators considered here are biased but comparatively $T_{NN}$ exhibits a smaller bias. $T_{NN}$ can be seen to be a very efficient estimator of the finite population total since it has smaller RMSE, followed closely by $T_{LP}$ and $T_{MARS}$. $T_{GAM}$ proved to be a very inefficient estimator of all other estimators.

Table 1. Unconditional bias, mean square error, relative root mean square error, mean absolute error and mean absolute percentage error for real data set.

The conditional performance of the estimator was done and compared with the performance of other existing population total estimators. To do this, 500 random samples, all of sizes 100 and 50, were selected and the mean of the auxiliary values xi was computed for each sample to obtain 200 values of $\bar{X}$. These sample means were then sorted in ascending order and further grouped into clusters of size 20 such that a total of 25 groups was realized. Further, group means of the means of auxiliary variables were calculated to get $\overline{\stackrel{¯}{X}}$. Empirical means and biases were then computed for all the estimators $T_{NN}$, $T_{LP}$, $T_{MARS}$ and $T_{GAM}$. The conditional biases were plotted against $\overline{\stackrel{¯}{X}}$ to provide a good understanding of the pattern generated. Figure 1 and Figure 2 show the behavior of the conditional biases, relative absolute biases and mean squared error realized by all the estimators based on the real data set.

In most cases, there are significant differences among the bias characteristics of the various estimators. A detailed examination of the plots reveals that $T_{NN}$ has lower levels of bias followed by $T_{LP}$ as indicated by the proximity of plotted curves to the horizontal (no bias) line at 0:0 on the vertical axis. Interestingly, despite the rather entangled nature of some of the plots, estimator $T_{NN}$ emerges clearly as the least biased for nearly every group means of the means of auxiliary variables.

Plots of Conditional MSE versus group means of the means of auxiliary variables similarly reveal coincident behavior for the estimators. $T_{NN}$ and $T_{LP}$ produce generally the lowest MSE values. In particular, $T_{NN}$ yields the lowest MSE in most cases among all other estimators. $T_{NN}$ is consistently better than all other estimators for both bias and MSE. All of these estimators are asymptotically unbiased and they all exhibit MSE consistency in that the MSE values tend toward zero as sample size increases. From the plots it can be seen that $T_{NN}$ and $T_{LP}$ performed equally better than all other estimators of the true population total functions.

Figure 1. Conditional bias, mean square error, relative root mean square error and mean absolute error based on real data with a sample size of 100.

Figure 2. Conditional bias, mean square error, relative root mean square error and mean absolute error based on real data with a sample size of 50.

5. Conclusion and Recommendations

In this paper, an estimator for Finite Population Total has been developed by employing a Feed Forward Back Propagation Neural Network technique in Non-parametric Regression. Asymptotic properties such as the Consistency and Mean Squared Error for the developed estimator have also been derived. When applied to dataset obtained from the United Nations Development Programme 2020 report, the findings indicate that the proposed estimator has the lowest bias and root mean square error values compared to other existing estimators. The developed estimator is considered to be effective in addressing the curse of dimensionality that makes Local Polynomials and Kernel Estimators ineffective when dealing with High Dimensional Data. It should be noted that the proposed estimator has been considered in the case of Simple Random Sampling Without Replacement (SRSWoR). An extension to other sampling techniques such Stratification may be done since they rely on SRSWoR, and it is hypothesised that efficiency will be improved compared to other existing estimators in literature.

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Chambers, R.L. and Dunstan, R. (1986) Estimating Distribution Functions from Survey Data. Biometrika, 73, 597-604. https://doi.org/10.1093/biomet/73.3.597
[2]	Wang, S.J. and Dorfman, A.H. (1996) A New Estimator for the Finite Population Distribution Function. Biometrika, 83, 639-652. https://doi.org/10.1093/biomet/83.3.639
[3]	Hansen, M.H., et al. (1987) Some History and Reminiscences on Survey Sampling. Statistical Science, 2, 180-190. https://doi.org/10.1214/ss/1177013352
[4]	Dorfman, A.H. (1992) Nonparametric Regression for Estimating Totals in Finite Populations. In: Proceedings of the Section on Survey Research Methods, American Statistical Association Alexandria, 622-625.
[5]	Otieno, R.O. and Mwalili, T.M. (2000) Nonparametric Regression Method for Estimating the Error Variance in Unistage Sampling.
[6]	Jay Breidt, F. and Opsomer, J.D. (2000) Local Polynomial Regression Estimators in Survey Sampling. Annals of Statistics, 28, 1026-1053. https://doi.org/10.1214/aos/1015956706
[7]	Hardle, W. and Linton, O. (1994) Applied Nonparametric Methods. In: Engle, R.F. and McFadden, D., Eds., Handbook of Econometrics, Vol. 4, Elsevier, Amsterdam, 2295-2339. https://doi.org/10.1016/S1573-4412(05)80007-8
[8]	Chambers, R.L., Dorfman, A.H. and Hall, P. (1992) Properties of Estimators of the Finite Population Distribution Function. Biometrika, 79, 577-582. https://doi.org/10.1093/biomet/79.3.577
[9]	Montanari, G.E. and Ranalli, M.G. (2003) On Calibration Methods for Design Based Finite Population Inferences. Bulletin of the International Statistical Institute, 60, 2 p.
[10]	Stone, C.J. (1982) Optimal Global Rates of Convergence for Nonparametric Regression. The Annals of Statistics, 10, 1040-1053. https://doi.org/10.1214/aos/1176345969
[11]	Bickel, P.J. and Li, B. (2007) Local Polynomial Regression on Unknown Manifolds. Institute of Mathematical Statistics, Beachwood, Lecture Notes—Monograph Series, 177-186. https://doi.org/10.1214/074921707000000148
[12]	Friedman, J.H. (1991) Multivariate Adaptive Regression Splines. The Annals of Statistics, 19, 1-67. https://doi.org/10.1214/aos/1176347963
[13]	Di Ciaccio, A. and Montanari, G.E. (2001) A Nonparametric Regression Estimator of a Finite Population Mean. In: Book Short Papers CLADAG, Istituto di Statistica, Università degli Studi di Palermo, Palermo, 173-176.
[14]	Opsomer, J.D., Jay Breidt, F., Moisen, G.G. and Kauermann, G. (2007) Model Assisted Estimation of Forest Resources with Generalized Additive Models. Journal of the American Statistical Association, 102, 400-409. https://doi.org/10.1198/016214506000001491
[15]	El-Housseiny, A.R. and Ziedan, D. (2014) Estimation of Population Total Using Nonparametric Regression Models. Advances and Applications in Statistics, 39, 37-59.
[16]	Barron, A.R. (1993) Universal Approximation Bounds for Superpositions of a Sigmoidal Function. IEEE Transactions on Information Theory, 39, 930-945. https://doi.org/10.1109/18.256500
[17]	Ken-Ichi, F. (1989) On the Approximate Realization of Continuous Mappings by Neural Networks. Neural Networks, 2, 183-192. https://doi.org/10.1016/0893-6080(89)90003-8
[18]	Cybenko, G. (1989) Approximation by Superpositions of a Sigmoidal Function. Mathematics of Control, Signals and Systems, 2, 303-314. https://doi.org/10.1007/BF02551274
[19]	White, H. (1990) Connectionist Nonparametric Regression: Multilayer Feed forward Networks Can Learn Arbitrary Mappings. Neural Networks, 3, 535-549. https://doi.org/10.1016/0893-6080(90)90004-5
[20]	Franke, J. and Neumann, M.H. (2000) Bootstrapping Neural Networks. Neural Computation, 12, 1929-1949. https://doi.org/10.1162/089976600300015204
[21]	Gene Hwang, J.T. and Ding, A.A. (1997) Prediction Intervals for Artificial Neural Networks. Journal of the American Statistical Association, 92, 748-757. https://doi.org/10.1080/01621459.1997.10474027
[22]	Barron, A.R. (1994) Approximation and Estimation Bounds for Artificial Neural Networks. Machine Learning, 14, 115-133. https://doi.org/10.1007/BF00993164
[23]	Asnaashari, A., McBean, E.A., Gharabaghi, B. and Tutt, D. (2013) Forecasting Watermain Failure Using Artificial Neural Network Modelling. Canadian Water Resources Journal, 38, 24-33. https://doi.org/10.1080/07011784.2013.774153
[24]	Grenander, U. and Ulf, G. (1981) Abstract Inference. Technical Report.
[25]	Geman, S. and Hwang, C.-R. (1982) Nonparametric Maximum Likelihood Estimation by the Method of Sieves. The Annals of Statistics, 10, 401-414. https://doi.org/10.1214/aos/1176345782
[26]	Franke, J. and Diagne, M. (2006) Estimating Market Risk with Neural Networks. Statistics & Decisions, 24, 233-253. https://doi.org/10.1524/stnd.2006.24.2.233
[27]	Shen, X.X., Jiang, C., Sakhanenko, L. and Lu, Q. (2019) Asymptotic Properties of Neural Network Sieve Estimators.
[28]	Serfling, R.J. (1980, 2000) Approximation Theorems of Mathematical Statistics. John Wiley & Sons, Inc., Hoboken. https://doi.org/10.1002/9780470316481
[29]	Serfling, R.J. (2009) Approximation Theorems of Mathematical Statistics, Volume 162. John Wiley & Sons, Hoboken.
[30]	Liang, F.M. and Kuk, Y.C.A. (2004) A Finite Population Estimation Study with Bayesian Neural Networks. Survey Methodology, 30, 219-234.