Asymptotic Consistency of the James-Stein Shrinkage Estimator

Alex Samuel Mungo, Victor Mooto Nawa

School of Natural Sciences, University of Zambia, Lusaka, Zambia.

DOI: 10.4236/ojs.2023.136044                

Abstract

The study explores the asymptotic consistency of the James-Stein shrinkage estimator obtained by shrinking a maximum likelihood estimator. We use Hansen’s approach to show that the James-Stein shrinkage estimator converges asymptotically to some multivariate normal distribution with shrinkage effect values. We establish that the rate of convergence is of order  and rate , hence the James-Stein shrinkage estimator is -consistent. Then visualise its consistency by studying the asymptotic behaviour using simulating plots in R for the mean squared error of the maximum likelihood estimator and the shrinkage estimator. The latter graphically shows lower mean squared error as compared to that of the maximum likelihood estimator.

Keywords

Asymptotic, Consistency, Convergence, Efficiency, Mean Squared Error, Shrinkage

1. Introduction

A shrinkage estimator is an estimator that, either explicitly or implicitly, incorporates the effects of shrinkage. In loose terms this means that a naive or raw estimate is improved by combining it with other information. The term relates to the notion that the improved estimate is made closer to the “true value” by the supplied information than the raw estimate. Shrinkage estimation is a technique used in inferential statistics to reduce the mean squared error (MSE) of a given estimator. The idea of shrinking an estimator came in 1956 when Stein [1] established that we can reduce the MSE of an estimator if we give up a little on bias. This means that given an estimator we can shrink it to obtain another estimator with lower MSE and the efficiency of the new estimator is desirable in the way it estimates the “true” parameter value. This works well when the number of parameters is more than two ( $p\ge 3$ ) called the “James-Stein classical condition”. When we shrink a maximum likelihood estimator (MLE) under the “James-Stein condition”, we obtain a new shrinkage estimator which is closer to the assumed true value compared to the MLE. The magnitude of the improvement depends on the distance between the “true” parameter value and the parametric restriction which yields a shrinkage target denoted by ${\tilde{\theta }}_n^o$ . With all these modifications and restrictions to achieve this estimator, we ask ourselves if this desirable shrinkage estimator is asymptotically consistent and efficient.

Literature on shrinkage estimators is a lot but we just mention a few of the most relevant contributions to our study. James and Stein [2] used shrinking techniques to come up with an estimator called the James-Stein shrinkage estimator (JSSE) which has lower squared risk loss compared to the MLE. Baranchik [3] showed that the positive part James-Stein shrinkage estimator has a lower risk than an ordinary JSSE. Berger [4] gives a discussion on selecting a minimax estimator of a multivariate normal mean by considering different types of James-Stein type estimators. Stein [5] used shrinking techniques to estimate the mean for a multivariate normal distribution. Carter and Ullah [6] constructed the sampling distribution and F-ratios for a James-Stein shrinkage estimator obtained by shrinking an ordinary least square (OLS) in regression models. George [7] proposed a new minimax multiple shrinkage estimator that allows multiple specifications for selection of a set of targets to shrink a given estimator. Geyer [8] looked at the asymptotics of constrained M-estimators which also fall in the class of shrinkage estimators. Then Hansen [9] constructed a generalised James-Stein shrinkage estimator obtained by shrinking an MLE, and Hansen [10] derived its asymptotic distribution and showed that we can shrink towards a sub-parameter space.

2. Preliminaries

The theory of shrinkage techniques plays an important role in developing efficient statistical estimators which play a key role in statistical decision theory. Therefore, a clear understanding of the asymptotic behaviour of the James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ provides knowledge on the stability and efficiency of the estimator when the sample size value n grows without bound.

This paper will investigate the asymptotic consistency and efficiency of the James-Stein shrinkage estimator (JSSE) obtained by shrinking a maximum likelihood estimator (MLE) when we have observed variables $X~N_p\left(\theta \mathrm{,}\Sigma \right)$ . Though the shrinkage estimator we are interested in is biased its study is important because there is a realisation that efficiency (lower risk) dominates all other properties in estimation. Efron [11] discusses how bias dominates unbiasedness in estimation.

We proceed by considering the asymptotic distributions of all the three estimators important to this study by considering results in Hansen 2016. Using the asymptotic distribution derived by Hansen [10] , we employ Taylor’s theorem and some limit theorems to show that ${\hat{\beta }}_o^{\mathrm{<mo>\; *\; </mo>}}{\to }_p{\theta }_o$ the “true” parameter value as $n\to \infty$ . Then we evaluate the asymptotic distributional bias (ADB) for the estimators ${\hat{\beta }}_n$ , ${\tilde{\theta }}_n^o$ and ${\hat{\beta }}_o^{\mathrm{<mo>\; *\; </mo>}}$ and show that the variance of the latter achieves the Cramér-Rao bound (CRB) as $n\to \infty$ . The analysis is done along the sequences ${\theta }_n$ as $n\to \infty$ . Simulation plots are produced in a statistical package R to compare the JSSE and MLE in terms of mean squared error (MSE), consistency and convergence.

The paper is organised as follows. Section 2.1 presents the parametric set up. Section 2.2 gives the form of the JSSE considered in the study while Section 2.3 discusses the asymptotic distributions of the estimators. In Section 3 we present the main results by first presenting a lemma on the convergence in probability of the shrinking factor as Section 3.1. We then show the consistency of the shrinkage estimator in Theorem 1. In Section 3.2 we evaluate the ADBs of the estimators in play. Then we show that the James-Stein shrinkage estimator is asymptotically efficient in Section 3.3 and also establish the rate of convergence in Section 3.4. In Section 3.5 we present MSE plots comparing the JSSE and MLE and in Section 4 we give a discussion and analysis of the whole study. Then conclude our study by stating the main results.

The following definitions are used to establish the asymptotic consistency and efficiency of the James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ .

Definition 1

An estimator $T_n=h\left(X_1\mathrm{,}\cdots \mathrm{,}{X}_n\right)$ is said to be consistent for ${\theta }_n$ if it converges in probability to ${\theta }_n$ . That is, if for all $\epsilon >0$

$\underset{n\to \infty }{\lim }Pr\left(\left|T_n-{\theta }_n\right|<\epsilon \right)=1$

or

$\underset{n\to \infty }{\lim }Pr\left(\left|T_n-{\theta }_n\right|>\epsilon \right)=0$

where n is the sample size value.

Definition 2

Let $X_1\mathrm{,}\cdots \mathrm{,}{X}_n$ be independent and identically distributed (iid) according to a probability density $f_{\theta }\left(X\right)$ satisfying suitable regularity conditions. Suppose that $T_n=h\left(X_1\mathrm{,}\cdots \mathrm{,}{X}_n\right)$ is asymptotically normal say that $\sqrt{n}\left(T_n-{\theta }_n\right){\to }_p{N}_p\left(\mathrm{0,}{\Sigma }_n\right)$ for a positive definite matrix ${\Sigma }_n$ where $T_n$ is estimating ${\theta }_n$ . Then a sequence of estimators $\left\{T_n\right\}=\left\{h\left(X_1\mathrm{,}\cdots \mathrm{,}{X}_n\right)\right\}$ satisfying

$\underset{n\to \infty }{lim}\left[nVar\left(T_n\right)\right]=J_n{\left(\theta \right)}^{-1}$

for the fisher information $J_n\left(\theta \right)$ is said to be asymptotically efficient.

We now consider a statistical model. We describe the set up of the parameter of interest and the shrinking strategy used in the study.

2.1. Parametric Structure

Consider an unbiased estimator ${\hat{\theta }}_n$ for $\theta \in \Omega$ such that ${\hat{\theta }}_n~N_p\left(\theta \mathrm{,}{\Sigma }_n\right)$ is a p-multivariate normal, where elements of Ω are p-dimensional parameter vectors. Let ${\tilde{\theta }}_n^o$ (shrinkage target) be a restricted maximum likelihood estimator (RMLE) for $\theta \in {\Omega }_o$ a sub-parameter space partitioned from the whole parameter space Ω by a parametric restriction ${\Omega }_o=\left\{\theta \in \Omega \mathrm{<mo>\; :\; </mo>}a\left(\theta \right)=0\right\}$ where $a\left(\theta \right)\mathrm{<mo>\; :\; </mo>}{ℝ}^p\to {ℝ}^m$ . The sub-parameter space ${\Omega }_o$ provides a simple model of interest to shrink to. If $m=p$ then ${\Omega }_o=\left\{0\right\}$ ( $\theta$ is the kernel of ${ℝ}^p$ ) a singleton zero vector and if $m<p$ , we create a sub-model of particular interest.

Let $A\left(\theta \right)=\frac{\partial }{\partial \theta }{a}^{\prime }$ where $A$ is a shrinkage matrix of dimension $p\times m$ . We introduce another matrix $G$ which harmonises the dimension of the RMLE from m to p. Hence we have a mapping $g\left(\theta \right)\mathrm{<mo>\; :\; </mo>}{ℝ}^m\to {ℝ}^p$ such that $G\left(\theta \right)=\frac{\partial }{\partial \theta }{g}^{\prime }$ . The

matrix $G$ is an $m\times p$ matrix when we consider a sub-parameter space ${\Omega }_o$ and $G=I_p$ is a p-dimensional identity matrix when we have the whole parameter space Ω. We note that the matrix $G$ is used to increase the dimension of the RMLE since it will be m-dimensional. Therefore we have a plug-in restricted maximum likelihood estimator $g\left({\tilde{\theta }}_n^o\right)={\tilde{\beta }}_n^o$ . The matrix $G$ is a matrix harmonising the dimension due to shrinkage and the actual dimension of the parameters of interest p. The plug-in unrestricted MLE ${\hat{\theta }}_n$ in the shrinkage sense is denoted by ${\hat{\beta }}_n$ . With all parameters set, we present the generalised James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ in the next section.

2.2. Positive Part James-Stein Shrinkage Estimator

Let ${\hat{\theta }}_n$ be the MLE for $\theta \in \Omega$ and ${\tilde{\theta }}_n^o$ be a restricted maximum likelihood estimator for $\theta \in {\Omega }_o$ a sub-parameter space of the whole parameter space Ω such that the elements of the parameter space are in ${ℝ}^p$ as described before. Let $g\left({\tilde{\theta }}_n^o\right)={\hat{\beta }}_n^o$ be the plug-in estimator of the RMLE of p-dimension. Then the James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ obtained by shrinking the MLE towards the target ${\tilde{\theta }}_n^o$ is given by

${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}={\hat{\beta }}_n-{\left(\frac{p-2}{n{\left({\hat{\beta }}_n-{\tilde{\beta }}_n^o\right)}^{{\rm T}}{\Sigma }^{-1}\left({\hat{\beta }}_n-{\tilde{\beta }}_n^o\right)}\right)}_{+}\left({\hat{\beta }}_n-{\tilde{\beta }}_n^o\right)$ (1)

where ${\left(a\right)}_{+}$ is positive trimming function and $p\ge 3$ . The shrinkage estimator in (1) can be expressed as a weighted average by letting

$D_n=n{\left({\hat{\beta }}_n-{\tilde{\beta }}_n^o\right)}^{{\rm T}}{\Sigma }^{-1}\left({\hat{\beta }}_n-{\tilde{\beta }}_n^o\right)$ (2)

a distance statistic which is the same as the loss function $n\mathcal{l}\left({\hat{\beta }}_n\mathrm{,}{\tilde{\beta }}_n^o\right)$ where $\Sigma$ is a covariance matrix and

$\hat{w}={\left(1-\frac{\tau }{D_n}\right)}_{+}$ (3)

for $\tau =p-2$ . Then (1) becomes

${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}=\hat{w}{\hat{\beta }}_n+\left(1-\hat{w}\right){\tilde{\beta }}_n^o$ (4)

which is a weighted average of ${\hat{\beta }}_n$ and ${\tilde{\beta }}_n^o$ . The James-Stein shrinkage estimator presented above has lower risk compared to the MLE as shown by Hansen [10] and James and Stein [2] . To check whether the shrinkage estimator is asymptotically consistent we need its asymptotic distribution. We therefore present the asymptotic distributions for the MLE, RMLE and JSSE in the next section.

2.3. Asymptotic Distribution

We assume that the maximum likelihood estimator satisfies the assumptions for regularity conditions given in Hansen [10] and Newey [12] . With these assumptions in mind, the asymptotic distributions of ${\tilde{\beta }}_n^o$ and ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ are analysed along

the sequences ${\theta }_n={\theta }_o+n^{-\frac{1}{2}}h$ where ${\theta }_o$ is the assumed true parameter value and $h\in {ℝ}^p$ is a constant providing a neighbourhood for the true parameter value ${\theta }_o$ . From the normality of the MLE we have

$\sqrt{n}\left({\hat{\theta }}_n-{\theta }_n\right){\to }_{d}Z~N_p\left(0\mathrm{,}\Sigma \right)$ (5)

as $n\to \infty$ . Using (5), Hansen [10] obtained the asymptotic distribution of the restricted maximum likelihood estimator as

$\sqrt{n}\left({\tilde{\theta }}_n^o-{\theta }_n\right){\to }_{p}Z-\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}\left(Z+h\right)$ (6)

which has some shrinkage value effect $k=\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}$ . As a consequence of the convergence in (5) and (6) we have

$\sqrt{n}\left({\hat{\beta }}_n-{\tilde{\beta }}_n^o\right){\to }_d{G}^{{\rm T}}\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}\left(Z+h\right)$ (7)

which is an asymptotic distribution the MLE converges to when it is estimating the RMLE where ${\beta }_n=g\left({\theta }_n\right)$ . The distance statistic $D_n$ in Equation (2) converges to some distribution given by a non central chi-squared distribution as described by Hansen [10]

$D_n=n\mathcal{l}\left({\hat{\beta }}_n\mathrm{,}{\tilde{\beta }}_n^o\right){\to }_d{\left(Z+h\right)}^{{\rm T}}B\left(Z+h\right)=\xi ~{\chi }_p^2\left(h^{{\rm T}}Bh\right)$ (8)

where matrix $B=A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}\Sigma G{\Sigma }^{-1}{G}^{{\rm T}}\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}$ . Using (2) Hansen [10] showed that

$\hat{w}{\to }_{d}w\left(Z\right)={\left(1-\frac{p-2}{\xi }\right)}_{+}$ (9)

which is a positive asymptotic distribution of an inverse of a chi-squared distribution with constant $\tau =p-2$ for $p\ge 3$ . Therefore using (9) as $n\to \infty$ , Hansen [10] showed that

$\sqrt{n}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_n\right){\to }_{d}w\left(Z\right)G^{{\rm T}}Z+\left(1-w\left(Z\right)\right)\left(G^{{\rm T}}Z-G^{{\rm T}}\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}\left(Z+h\right)\right)$ (10)

which is normally distributed with some shrinkage value effect. With the asymptotic distribution of the shrinkage estimator in place we now present the main results.

3. Main Results

In this section we present our main results. We show that the James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ is asymptotically consistent. Secondly, we evaluate the asymptotic distribution bias of the three estimators in play. We then show that the shrinkage estimator is asymptotically efficient by showing that its variance achieves the Cramér-Rao bound. Further, we explore the convergence rate of the James-Stein shrinkage estimator and present the simulation plots for the MSE done in R.

3.1. Consistency of the James-Stein Shrinkage Estimator

We present Lemma 1 which shows the convergence in probability of the weight (shrinkage factor) $\hat{w}$ . The result is used when establishing the consistency of the James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ .

Lemma 1

From Equation (8) we have

$\hat{w}{\to }_{d}w\left(Z\right)={\left(1-\frac{\tau }{\xi }\right)}_{+}$ (11)

where $\xi ={\left(Z+h\right)}^{{\rm T}}B\left(Z+h\right)~{\chi }_p^2\left(h^{{\rm T}}Bh\right)$ is a non central Chi-squared distribution with non centrality parameter $h^{{\rm T}}Bh$ , $\tau =p-2$ and $p\ge 3$ . Along the sequences ${\theta }_n$ , if $h\to \infty$ then

$w\left(Z\right){\to }_{p}1$ (12)

and if $h$ is fixed then

$w\left(Z\right){\to }_{p}0$ otherwise $w\left(Z\right){\to }_{p}r$ (13)

where $r$ is a constant such that $0<r<1$ and ${\left(a\right)}_{+}$ in (11) is a positive trimming function which keeps what is in the brackets greater than or equal to zero.

Proof.

We begin by considering the first case when $h$ diverges to infinity. Suppose that $h\to \infty$ then

$\left(Z+h\right)\to \infty$ as $n\to \infty$ . (14)

Therefore from (14) we have

${\left(Z+h\right)}^{{\rm T}}B\left(Z+h\right)=\xi \to \infty$ as $h\to \infty$ and $n\to \infty$ . (15)

Now considering that

$\hat{w}{\to }_{d}w\left(Z\right)={\left(1-\frac{\tau }{\xi }\right)}_{+}$

as $n\to \infty$ . Using (15) we have

$w\left(Z\right){\to }_{p}1$ as $n\to \infty$ . (16)

Hence we have established (12).

Secondly suppose that $h$ is fixed, then we have the sequence

${\theta }_n={\theta }_o+n^{-\frac{1}{2}}h$

which becomes ${\theta }_n={\theta }_o$ as $n\to \infty$ implying that ${\hat{\theta }}_n\to {\theta }_o$ as $n\to \infty$ . Suppose

$\xi ={\chi }_p^2\left(h^{{\rm T}}Bh\right){\to }_{p}D$ as $n\to \infty$ (17)

where D is a constant, $h$ is fixed and $B$ is not affected by an increase in n, then

$w\left(Z\right){\to }_p{\left(1-\frac{\tau }{D}\right)}_{+}$

where $p\ge 3$ . If $\frac{\tau }{D}=1$ as $n\to \infty$ , then

$w\left(Z\right){\to }_{p}0.$

If $\frac{\tau }{D}>1$ then $\left(1-\frac{\tau }{D}\right)$ will be negative and by definition of the positive streaming function we end up with zero. This will vary as p changes but still considering $\xi ~{\chi }_p^2\left(h^{{\rm T}}Bh\right)$ the probability of $\xi$ depends on the degrees of freedom p and will vary according to the chi-squared distribution, implying that the ratio $\frac{\tau }{D}=M\ge 1$ as $n\to \infty$ . Therefore we have

$w\left(Z\right){\to }_p{\left(1-\frac{\tau }{D}\right)}_{+}$ as $n\to \infty$ ,

$w\left(Z\right){\to }_p{\left(1-M\right)}_{+}$ as $n\to \infty$

for a constant $M>1$ . Proceeding in the same way we have

$w\left(Z\right){\to }_p{\left(F\right)}_{+}$ as $n\to \infty$ , for $1-M=F<0$ ,

$w\left(Z\right){\to }_{p}0$ as $n\to \infty$

by definition of ${\left(x\right)}_{+}$ . Thus

$w\left(Z\right){\to }_{p}0$ as $n\to \infty$ . (18)

Otherwise, if the ratio $\frac{\tau }{D}$ is such that $0<\frac{\tau }{D}<1$ as $n\to \infty$ , we have

$w\left(Z\right){\to }_{p}r$

where $r\in \left(\mathrm{0,1}\right)$ .

Lemma 1 above establishes convergence of the weight $\hat{w}$ which determines shrinkage. In this case we realise that the same weight determines the convergence in distribution and probability of the shrinkage estimator. From the regularity conditions we know that the MLE ${\hat{\theta }}_n$ is asymptotically consistent and this consistency extends to the RMLE ${\bar{\theta }}_n^o$ . With this fact in mind, we now present the main result which shows the consistency of the shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ .

Theorem 1

Let $\theta \in \Omega$ , where Ω is a parameter space with elements in ${ℝ}^p$ . Suppose we have a James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ which is obtained by shrinking the maximum likelihood estimator ${\hat{\theta }}_n$ of $\theta \in \Omega$ where the shrinkage target ${\tilde{\theta }}_n^o$ is the restricted maximum likelihood estimator of $\theta \in {\Omega }_o$ a partitioned sub-parameter space from Ω by the restriction described in Section 2.1. Then the JSSE is given by

${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}={\hat{\beta }}_n-{\left(\frac{p-2}{n{\left({\hat{\beta }}_n-{\tilde{\beta }}_n^o\right)}^{{\rm T}}{\Sigma }^{-1}\left({\hat{\beta }}_n-{\tilde{\beta }}_n^o\right)}\right)}_{+}\left({\hat{\beta }}_n-{\tilde{\beta }}_n^o\right)$

where ${\hat{\beta }}_n=g\left({\hat{\theta }}_n\right)$ , ${\tilde{\beta }}_n^o=g\left({\tilde{\theta }}_n^o\right)$ , $p\ge 3$ and ${\left(x\right)}_{+}$ is a positive trimming function. If ${\hat{\theta }}_n$ is consistent for ${\theta }_n$ as $n\to \infty$ then the James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ is also consistent for ${\theta }_n$ as $n\to \infty$ , where the sequence ${\theta }_n$ is as defined in Section 2.3.

Proof.

Let ${\beta }_n={\theta }_n$ . To show that ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ is consistent for ${\theta }_n$ as $n\to \infty$ we consider the value of $h$ which determines the neighbourhood of the sequence ${\theta }_n$ , when it diverges to infinity and when it is just fixed. Suppose that $h$ diverges to infinity, to evaluate

$\sqrt{n}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_n\right){\to }_{d}w\left(Z\right)G^{{\rm T}}Z+\left(1-w\left(Z\right)\right)G^{{\rm T}}\left(Z-\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}\left(Z+h\right)\right)$ (19)

as $n\to \infty$ , we first consider $w\left(Z\right)$ from (3). By Lemma 1

$w\left(Z\right){\to }_{p}1$ as $n\to \infty$ . (20)

Hence from (20) and substituting $w\left(Z\right)$ by 1 in (19) we have

$\sqrt{n}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_n\right){\to }_d{G}^{{\rm T}}Z~G^{{\rm T}}{N}_p\left(0\mathrm{,}\Sigma \right)$

which gives

$\sqrt{n}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_n\right){\to }_d{N}_p\left(0\mathrm{,}{\Sigma }_{\beta }\right)$ (21)

as $n\to \infty$ where ${\Sigma }_{\beta }=G^{{\rm T}}\Sigma G$ . Thus we have

${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}{\to }_p{\beta }_n$

if

$\underset{n\to \infty }{\lim }P\left(\left|{\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_n\right|>\epsilon \right)=0$

for any $\epsilon >0$ . Hence ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ is consistent for ${\beta }_n={\theta }_n$ .

Secondly, suppose $h$ is fixed within an imaginable value, then the sequence

${\theta }_n={\theta }_o+n^{-\frac{1}{2}}h$

becomes ${\theta }_n={\theta }_o$ as $n\to \infty$ . From this equality we have ${\hat{\theta }}_n={\hat{\theta }}_o$ and two conditions arise. The first one is that the sequence ${\theta }_n$ will be within the restricted parameter space ${\Omega }_o$ with ${\theta }_o\in {\Omega }_o$ . From the restriction $\left\{\theta \in \Omega \mathrm{<mo>\; :\; </mo>}a\left(\theta \right)=0\right\}$ of ${\Omega }_o$ this means that the shrinkage target is exactly at the true value and our consideration will be just on one parameter space. Therefore, we have ${\hat{\theta }}_n={\tilde{\theta }}_n^o$ , but from (6)

$\sqrt{n}\left({\tilde{\theta }}_n^o-{\theta }_n\right){\to }_d\zeta =Z-\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}\left(Z+h\right)$

which will be the same as the asymptotic distribution of ${\hat{\theta }}_n$ since we only consider the sub-parameter space ${\Omega }_o$ , and the shrinkage value $\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}\left(Z+h\right)$ affects it. Thus

$\sqrt{n}\left({\hat{\theta }}_n-{\theta }_n\right){\to }_d\zeta =Z-\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}\left(Z+h\right)$ as $n\to \infty$ (22)

because we are estimating $\theta \in {\Omega }_o$ and from Section 2.1 there will be no difference between the MLE and RMLE. Due to this equality of the two maximum likelihood estimators, from (4)

${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}=\hat{w}{\hat{\beta }}_n+\left(1-\hat{w}\right){\tilde{\beta }}_n^o$

and substituting (22) in (19) we have

$\begin{array}{l}\sqrt{n}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_n\right){\to }_{d}w\left(Z\right)G^{{\rm T}}\left(Z-\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}\left(Z+h\right)\right)\\ +\left(1-w\left(Z\right)\right)G^{{\rm T}}\left(Z-\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}\left(Z+h\right)\right)\end{array}$

as $n\to \infty$ , which becomes

$\begin{array}{c}\sqrt{n}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_n\right){\to }_d\left[w\left(Z\right)G^{{\rm T}}\left(Z-\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}\left(Z+h\right)\right)\right]\\ -\left[w\left(Z\right)G^{{\rm T}}\left(Z-\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}\left(Z+h\right)\right)\right]\\ +G^{{\rm T}}\left(Z-\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}\left(Z+h\right)\right)\end{array}$

as $n\to \infty$ , and then simplifies to

$\sqrt{n}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_n\right){\to }_d{G}^{{\rm T}}\left(Z-\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}\left(Z+h\right)\right)$ (23)

as $n\to \infty$ , which is the same as the asymptotic distribution of ${\tilde{\beta }}_n^o=g\left({\tilde{\theta }}_n^o\right)$ . Therefore using the consistency of the RMLE and (23), the consistency of the James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ follows from the consistency of ${\tilde{\theta }}_n^o$ .

Lastly, we consider the case when we have two well defined parameter spaces, ${\Omega }_o$ and $\Omega -{\Omega }_o$ . Then we have ${\hat{\theta }}_n\ne {\tilde{\theta }}_n^o$ . Analysing (19) further, we consider the shrinkage effect value $\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}$ which is not affected by the sample size value n but it is a value affected by an increase or decrease in the number of parameters p. Since

$Z~N_p\left(0\mathrm{,}\Sigma \right)$

then

$Z+h~N_p\left(h\mathrm{,}\Sigma \right)$ (24)

by linearity property of the normal distribution. Also implying that

$\eta \left(Z+h\right)~N_p\left(\eta h\mathrm{,}{\eta }^{{\rm T}}\Sigma \eta \right)$ (25)

for some matrix $\eta$ of dimension $p\times p$ . From (13) of Lemma 1 we have

$w\left(Z\right){\to }_{p}0$ if $\frac{\tau }{D_n}\ge 1$ (26)

as $n\to \infty$ . Therefore (19) becomes

$\sqrt{n}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_n\right){\to }_{d}w\left(Z\right)G^{{\rm T}}Z+\left(1-w\left(Z\right)\right)G^{{\rm T}}\left(Z-\eta \left(Z+h\right)\right)$

for some shrinkage value effect matrix $\eta =\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}$ . Evaluating this asymptotic distribution as $n\to \infty$ we have

$w\left(Z\right)G^{{\rm T}}Z+\left(1-w\left(Z\right)\right)G^{{\rm T}}\left(Z-\eta \left(Z+h\right)\right){\to }_d{G}^{{\rm T}}\left(Z-\eta \left(Z+h\right)\right)$

as $n\to \infty$ since $w\left(Z\right){\to }_{p}0$ . Thus

$\sqrt{n}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_n\right){\to }_d{G}^{{\rm T}}\left(Z-\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}\left(Z+h\right)\right)$ as $n\to \infty$ .

Hence the consistency of ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ follows from the consistency of ${\tilde{\beta }}_n^o=g\left({\tilde{\theta }}_n^o\right)$ which is consistent since ${\tilde{\theta }}_n^o$ is consistent. Similarly if $w\left(Z\right)\to r\in \left(\mathrm{0,1}\right)$ , the consistency of the James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ follows from the consistency of the restricted maximum likelihood estimator and also the fact that ${\hat{\theta }}_n$ is consistent for ${\theta }_n$ . Thus the shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ is asymptotically consistent for ${\theta }_n$ .

In Theorem 1, we first consider the case when the sequence ${\theta }_n$ has a neighbourhood which is not restricted by letting $h$ diverge to infinity. When this is the case, the entire parameter space becomes of interest and for $h\to \infty$ we obtain $\xi {\to }_p\infty$ and $\hat{w}{\to }_{p}1$ as $n\to \infty$ . Hence there is no difference on how the parameters in ${\Omega }_o$ and Ω are asymptotically distributed. As a result the asymptotic distribution of the James-Stein shrinkage estimator is the same as that of the initial maximum likelihood estimator under this condition. Therefore the consistency of the James-Stein shrinkage estimator follows from the consistency of the maximum likelihood estimator.

In the second case we take $h$ as a fixed imaginable value. In this case the two parameter spaces are well defined and distinctive in terms of where the parameters of interest are located. When $n\to \infty$ then ${\theta }_n={\theta }_o$ since $n^{-\frac{1}{2}}h\to 0$ . Thus

when we are within the restricted sub-parameter space ${\Omega }_o$ the maximum likelihood estimator and the restricted likelihood estimator are asymptotically distributed the same. The consequence of having the two maximum likelihood estimators (MLE and RMLE) distributed the same results in the James-Stein shrinkage estimator having the same asymptotic distribution as the MLE and RMLE. Furthermore, we have $\sqrt{n}\left({\hat{\beta }}_n-{\tilde{\beta }}_n^o\right){\to }_{p}0$ as $n\to \infty$ . Stone [13] obtained similar results though under invariant estimators. In this case the two maximum likelihood estimators do not have to be necessarily invariant. Therefore the James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ is asymptotically consistent for ${\theta }_n$ .

In the next section we investigate the asymptotic distribution bias of ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ . The results in this section are used in showing the asymptotic efficiency of the shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ .

3.2. Asymptotic Distributional Bias

We study the asymptotic distributional bias (ADB) for the three estimators by analysing the asymptotic bias values. The ADB of an estimator $T_n$ is given by

$ADB\left(T_n\right)=\underset{n\to \infty }{\lim }E\left[\sqrt{n}\left(T_n-{\theta }_n\right)\right]$ (27)

where the estimator $T_n$ is estimating ${\theta }_n$ . We present the asymptotic distributional bias for ${\hat{\theta }}_n$ , ${\tilde{\theta }}_n^o$ and ${\hat{\beta }}_n$ in the theorem below.

Theorem 2

Suppose that regularity assumptions for the MLE and RMLE hold. Then under $\left\{P_n\right\}$ a sequence of parameter dimension with the sample size value n and $p\ge 3$ , the ADBs of the estimators ${\hat{\theta }}_n$ , ${\tilde{\theta }}_n^o$ and ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ are respectively

1. $ADB\left({\hat{\theta }}_n\right)=0$

2. $ADB\left({\tilde{\theta }}_n^o\right)=-\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}h$

3. $ADB\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)=-\vartheta G^{{\rm T}}\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}h$

where $\vartheta =E_{\theta }\left[\frac{p-2}{\xi }\right]$ .

Proof.

1.

$\begin{array}{l}ADB\left({\hat{\theta }}_n\right)=\underset{n\to \infty }{\lim }{E}_{\theta }\left(\sqrt{n}\left({\hat{\theta }}_n-{\theta }_n\right)\right)\\ =\underset{n\to \infty }{\lim }0\\ =0\\ \therefore ADB\left({\hat{\theta }}_n\right)=0\end{array}$ (28)

since $\sqrt{n}\left({\hat{\theta }}_n-{\theta }_n\right){\to }_{d}Z~N_p\left(0\mathrm{,}\Sigma \right)$ as $n\to \infty$ .

2.

$\begin{array}{l}ADB\left({\tilde{q}}_n^o\right)=\underset{n\to \infty }{\lim }{E}_{\theta }\left(\sqrt{n}\left({\tilde{\theta }}_n^o-{\theta }_n\right)\right)\\ =\underset{n\to \infty }{\lim }\left(-\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}h\right)\\ =-\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}h\\ \therefore ADB\left({\tilde{q}}_n^o\right)=-\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}h\end{array}$ (29)

from Equation (6).

3. $ADB\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)=\underset{n\to \infty }{\lim }{E}_{\theta }\left(\sqrt{n}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_n\right)\right)\mathrm{.}$ From Equation (19) of Theorem 1 we have

$\sqrt{n}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_n\right){\to }_{d}w\left(Z\right)G^{{\rm T}}Z+\left(1-w\left(Z\right)\right)G^{{\rm T}}\left(Z-\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}\left(Z+h\right)\right)$

where

$w\left(Z\right)={\left(1-\frac{p-2}{\xi }\right)}_{+}\mathrm{.}$

Therefore,

$\begin{array}{c}{E}_{\theta }\left[w\left(Z\right)\right]=\left[1-\left(\frac{p-2}{\xi }\right)\right]\\ =1-E_{\theta }\left[\frac{p-2}{\xi }\right]\\ =1-\vartheta \end{array}$

where $\vartheta =E_{\theta }\left[\frac{p-2}{\xi }\right]$ , $p\ge 3$ and $E_{\theta }\left(Z\right)=0$ as $n\to \infty$ since $Z~N_p\left(0\mathrm{,}\Sigma \right)$ .

Then

$\begin{array}{l}ADB\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)=\underset{n\to \infty }{{\displaystyle \lim }}\left[\left(1-\vartheta \right)0+\left(1-1+\vartheta \right)\left(-G^{{\rm T}}\left(\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}h\right)\right)\right]\\ =\underset{n\to \infty }{\lim }\left[-\vartheta G^{{\rm T}}\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}h\right]\\ =-\vartheta G^{{\rm T}}\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}h\\ \therefore ADB\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)=-\vartheta G^{{\rm T}}\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}h\end{array}$ (30)

where $\vartheta =E_{\theta }\left[\frac{p-2}{\xi }\right]$ for $p\ge 3$ .

Remark 1

When the fixed constant $h=0$ , the asymptotic distributional bias values of the three estimators are zero. Therefore $h\ne 0$ .

From Equation (28) of Theorem 2, the maximum likelihood estimator is asymptotically unbiased. Equations (29) and (30) show that the restricted maximum likelihood estimator and the James-Stein shrinkage estimator are both asymptotically biased. This means that both shrinking and partitioning of a parameter space brings bias to estimators.

Using the bias of the shrinkage estimator obtained above, we now analyse whether the James-Stein estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ is asymptotically efficient.

3.3. Asymptotic Efficient

To check whether the shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ is asymptotically efficient, we use the Cramér-Rao bound for biased estimators. In the theorem below we show that the variance of the JSSE achieves this bound as $n\to \infty$ . We use concepts of the study by Hogdes and Lehman [14] .

Theorem 3

Let ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ be a James-Stein shrinkage estimator obtained by shrinking a maximum likelihood estimator ${\hat{\theta }}_n$ where the two estimators are as defined in Section 2.2. Given the asymptotic bias $b_{\theta }\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)$ of the JSSE ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ , the Cramér-Rao bound for ${\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}$ is given by

$CRB=\frac{{\left[1+{b^{\prime }}_{{\theta }_j}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)\right]}^2}{J_{jj}\left(\theta \right)}$ for $j=\mathrm{1,2,}\cdots ,p$ (31)

where $J\left(\theta \right)$ is the fisher information and ${b^{\prime }}_{{\theta }_j}$ is the derivative of the j^th element of the bias vector. Then

$\frac{CRB}{{\Sigma }_{jj}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)}=1$ as $n\to \infty$ ,

and thus the James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ is asymptotically efficient for all $j=\mathrm{1,2,}\cdots ,p$ .

Proof. We analyse asymptotic efficiency by evaluating the Cramér-Rao bound as $n\to \infty$ . Consider the bias of the estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ from part 3 of Theorem 2

$b_{\theta }\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)=-\vartheta G^{{\rm T}}\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}h$ (32)

where $\vartheta =E\left[\frac{p-2}{\xi }\right]$ for $p\ge 3$ . The expectation $E\left[\frac{p-2}{\xi }\right]$ of the fraction $\frac{p-2}{\xi }$ which follows a distribution determined by the distribution $\xi ~{\chi }_p^2\left(h^{\prime }Bh\right)$ has a value (constant) free of the parameter $\theta$ . Therefore we regard it as a constant. Let $\alpha =-\vartheta$ then (32) becomes

$b_{\theta }\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)=\alpha G^{{\rm T}}\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}h$ (33)

and ${b^{\prime }}_{\theta }\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)$ will be

$\begin{array}{c}{b^{\prime }}_{\theta }\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)=\frac{\partial }{\partial \theta }{b}_{\theta }\mathrm{<mo\; stretchy="false">\; (\; </mo>}{\hat{b}}_n^{\mathrm{<mo>\; *\; </mo>}}\mathrm{<mo\; stretchy="false">\; )\; </mo>}\\ =\alpha \frac{\partial }{\partial \theta }\left(G^{{\rm T}}\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}h\right)\end{array}$ (34)

a matrix of dimension p. Using the definition of the CRB and then combining (32) and (34) we obtain

$\frac{{\left[1+{b^{\prime }}_{{\theta }_j}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)\right]}^2}{J_{jj}\left(\theta \right)}=\frac{{\left[1+\alpha \frac{\partial }{\partial {\theta }_j}\left(G^{{\rm T}}\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}h\right)\right]}^2}{J_{jj}\left(\theta \right)}$ (35)

for $j=\mathrm{1,2,}\cdots ,p$ where $\frac{\partial }{\partial {\theta }_j}$ is the partial derivative of the j^th element, $\Sigma =\Sigma \left(\theta \right)={\left(-{\displaystyle \underset{i=1}{\overset{n}{\sum }}}\frac{{\partial }^2}{\partial \theta \partial {\theta }^{\prime }}\log f_{\theta }\left(X_i\right)\right)}^{-1}$ , $A=A\left({\theta }_o\right)=\frac{\partial }{\partial \theta }a{\left(\theta \right)}^{\prime }$ and

$J=J\left(\theta \right)=\mathbb{E}\left[{\displaystyle \underset{i=1}{\overset{n}{\sum }}}-\frac{{\partial }^2}{\partial \theta \partial {\theta }^{\prime }}\log f_{\theta }\left(X_i\right)\right]\mathrm{.}$

We begin our analysis of the bound by considering the terms involved. Thus

$\frac{\partial }{\partial \theta }A=\frac{{\partial }^2}{\partial \theta \partial {\theta }^{\prime }}a{\left(\theta \right)}^{\prime }$

remains the same as $n\to \infty$ . We have

$\Sigma \left(\theta \right)={\left(-{\displaystyle \underset{i=1}{\overset{n}{\sum }}}\frac{{\partial }^2}{\partial \theta \partial {\theta }^{\prime }}\log f_{\theta }\left(X_i\right)\right)}^{-1}\to {\Sigma }_p$ (36)

as $n\to \infty$ where the elements of ${\Sigma }_p$ are zeros apart from the diagonal elements ${\Sigma }_{jj}\left(\theta \right)$ which are ones for $j=\mathrm{1,2,}\cdots ,p$ since the observations are iid and follow a p-multivariate standard normal distribution. Thus from (36) we have

$\frac{\partial }{\partial {\theta }_j}{\Sigma }_{jj}\left(\theta \right)\to 0$ and $\frac{\partial }{\partial \theta }\Sigma \to 0$ (37)

as $n\to \infty$ for $j=\mathrm{1,2,}\cdots ,p$ . This implies that

$\frac{\partial }{\partial {\theta }_j}{G}^{{\rm T}}\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}h\to 0$ and $\frac{\partial }{\partial \theta }{G}^{{\rm T}}\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}h\to 0$ (38)

for $j=\mathrm{1,2,}\cdots ,p$ as $n\to \infty$ . Then from (38) we have

${b^{\prime }}_{{\theta }_j}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)\to 0$ and ${b^{\prime }}_{\theta }\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)\to 0$ (39)

for $j=\mathrm{1,2,}\cdots ,p$ as $n\to \infty$ . Therefore from (38) and (39), then using (35) we have

$\begin{array}{c}\frac{{\left[1+{b^{\prime }}_{{\theta }_j}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)\right]}^2}{J_{jj}\left(\theta \right)}=\frac{{\left[1+\alpha \frac{\partial }{\partial {\theta }_j}\left(G^{{\rm T}}\Sigma A{\left(A^{{\rm T}}\Sigma A\right)}^{-1}{A}^{{\rm T}}h\right)\right]}^2}{J_{jj}\left(\theta \right)}\\ =\frac{{\left[1+0\right]}^2}{J_{jj}\left(\theta \right)}\\ =J_{jj}{\left(\theta \right)}^{-1}\end{array}$ (40)

as $n\to \infty$ for $j=\mathrm{1,2,}\cdots ,p$ . Since for all $j=\mathrm{1,2,}\cdots ,p$ we have ${\Sigma }_{jj}\left(\theta \right)=J_{jj}{\left(\theta \right)}^{-1}$ then $\Sigma =J^{-1}$ as $n\to \infty$ . Hence from (40) we have the variance for the James-Stein shrinkage estimator ${\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}$ , ${\Sigma }_{jj}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)$ converges to the CRB as $n\to \infty$ for all $j=\mathrm{1,2,}\cdots ,p$ . This means that

$\frac{CRB}{{\Sigma }_{jj}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)}\to \frac{{\Sigma }_{jj}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)}{{\Sigma }_{jj}\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)}=1$ as $n\to \infty$

for all $j=\mathrm{1,2,}\cdots ,p$ . Thus the James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ is asymptotically efficient.

Theorem 3 above shows that the James-Stein shrinkage estimator obtained from shrinking the MLE achieves the CRB asymptotically. This means that the shrinkage estimator is asymptotically efficient and hence stable when we have large sample size values. Since the initial estimator (MLE) is known to be asymptotically efficient then we see that the shrinking process has no effect on the asymptotic efficiency of an estimator we are shrinking.

3.4. Rate of Convergence

We now investigate the rate of convergence of the shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ (JSSE) by using concepts applied on the MLE discussed in Hoeffding [15] . To proceed we consider the shrinkage estimator of the form in (1) using plug-in maximum likelihood estimators ${\hat{\beta }}_n$ and ${\tilde{\beta }}_n^o$ . Let ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ be a James-Stein shrinkage estimator obtained when we shrink the MLE ${\hat{\beta }}_n=g\left({\hat{\theta }}_n\right)$ defined earlier before for $p\ge 3$ . We proceed to find the rate of convergence of this estimator by using its relationship with the MLE. Since the shrinkage target value may have no effect on the convergence rate, for easier transformation of our sequence ${\theta }_n$ we set ${\tilde{\theta }}_n^o=0$ implying ${\tilde{\beta }}_n^o=0$ . Thus we have

${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}={\hat{\beta }}_n-{\left(\frac{p-2}{n\left({\hat{\beta }}_n^{{\rm T}}{V}^{-1}{\hat{\beta }}_n\right)}\right)}_{+}{\hat{\beta }}_n$

which becomes

${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}={\left(1-\frac{p-2}{{\hat{\beta }}_n^{{\rm T}}{V}^{-1}{\hat{\beta }}_n}\right)}_{+}{\hat{\beta }}_n$

when we factor out ${\hat{\beta }}_n$ and drop out the n in the denominator to have a form with a lower MSE according to the James-Stein shrinkage strategy. Let

$k={\left(1-\frac{p-2}{{\hat{\beta }}_n^{{\rm T}}{V}^{-1}{\hat{\beta }}_n}\right)}_{+}\mathrm{,}$ (41)

then

${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}=k{\hat{\beta }}_n\mathrm{.}$ (42)

Now consider the sequence

${\hat{\beta }}_{n_j}={\beta }_{o_j}+O_p\left(\frac{1}{\sqrt{n}}\right)$ (43)

for $j=\mathrm{1,2,}\cdots ,p$ where ${\beta }_{o_j}$ is the “true” j^th parameter value. From the equality in (42) we have

${\hat{\beta }}_n=\frac{1}{k}{\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ (44)

for the shrinkage value k. Therefore substituting the right hand side of (44) in (43) the sequence ${\hat{\beta }}_{n_j}$ becomes

$\frac{1}{k}{\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}={\beta }_{o_j}+O_p\left(\frac{1}{\sqrt{n}}\right)\mathrm{,}$

hence we have the sequence

${\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}=k{\beta }_{o_j}+O_p\left(\frac{1}{\sqrt{n}}\right)k$ (45)

which is in terms of the shrinkage estimator with the shrinking effect value k such that $0<k\le 1$ . Analysing this sequence further shows that it satisfies the smoothness regularity conditions for the MLE, therefore we can proceed.

Let ${\beta }_{o_j}^{\mathrm{<mo>\; *\; </mo>}}=k{\beta }_{o_j}$ be the true value in the shrinkage sense which is obtained when we scale the true value ${\beta }_{o_j}$ with the shrinkage factor k. Then the sequence (45) becomes

${\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}={\beta }_{o_j}^{\mathrm{<mo>\; *\; </mo>}}+O_p\left(\frac{1}{\sqrt{n}}\right)k$ (46)

implying that

${\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_{o_j}^{\mathrm{<mo>\; *\; </mo>}}=O_p\left(\frac{1}{\sqrt{n}}\right)k$ (47)

for all $j=\mathrm{1,2,}\cdots ,p$ . This means that $\left({\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_{o_j}^{\mathrm{<mo>\; *\; </mo>}}\right)$ is still within the neighbourhood of $\frac{1}{\sqrt{n}}$ since $0<k\le 1$ . Therefore, using the second order Taylor’s theorem we have

$\ln \frac{{\displaystyle \underset{i=1}{\overset{n}{\prod }}}{f}_{{\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}\left(x_i\right)}}{{\displaystyle \underset{i=1}{\overset{n}{\prod }}}{f}_{{\beta }_{o_j}^{\mathrm{<mo>\; *\; </mo>}}\left(x_i\right)}}=\left({\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_{o_j}^{\mathrm{<mo>\; *\; </mo>}}\right)\sqrt{n}\sqrt{I_{jj}\left({\beta }_o^{\mathrm{<mo>\; *\; </mo>}}\right)}{Z}_j-\frac{n}{2}{\left({\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_{o_j}^{\mathrm{<mo>\; *\; </mo>}}\right)}^2{I}_{jj}\left({\beta }_o^{\mathrm{<mo>\; *\; </mo>}}\right)+O_p\left(1\right)$ (48)

for $j=\mathrm{1,2,}\cdots ,p$ . Since

$\frac{\partial }{\partial \beta }\ln L\left({\hat{\beta }}_n\right)=0$

for the maximum likelihood estimator ${\hat{\beta }}_n$ , then also

$\frac{\partial }{\partial \beta }\ln L\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)=0$

implying that

$\frac{\partial }{\partial {\beta }_j}\ln L\left({\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}\right)=0$ (49)

for all $j=\mathrm{1,2,}\cdots ,p$ . Assuming that the log-likelihood function is differentiable, from (48) and (49) we have

$\begin{array}{l}\frac{\partial }{\partial {\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}}\ln \left(\frac{{\displaystyle \underset{i=1}{\overset{n}{\prod }}}{f}_{{\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}\left(x_i\right)}}{{\displaystyle \underset{i=1}{\overset{n}{\prod }}}{f}_{{\beta }_{o_j}^{\mathrm{<mo>\; *\; </mo>}}\left(x_i\right)}}\right)\\ ={\left({\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_{o_j}^{\mathrm{<mo>\; *\; </mo>}}\right)}^{1-1}\sqrt{n}\sqrt{I_{jj}\left({\beta }_o^{\mathrm{<mo>\; *\; </mo>}}\right)}{Z}_j-\frac{2n}{2}{\left({\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_{o_j}^{\mathrm{<mo>\; *\; </mo>}}\right)}^{2-1}{I}_{jj}\left({\beta }_o^{\mathrm{<mo>\; *\; </mo>}}\right)+O_p\left(1\right)\end{array}$ (50)

and simplifying (50) becomes

$\frac{\partial }{\partial {\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}}\ln \left(\frac{{\displaystyle \underset{i=1}{\overset{n}{\prod }}}{f}_{{\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}\left(x_i\right)}}{{\displaystyle \underset{i=1}{\overset{n}{\prod }}}{f}_{{\beta }_{o_j}^{\mathrm{<mo>\; *\; </mo>}}\left(x_i\right)}}\right)=\sqrt{n}\sqrt{I_{jj}\left({\beta }_o^{\mathrm{<mo>\; *\; </mo>}}\right)}{Z}_j-n\left({\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_{o_j}^{\mathrm{<mo>\; *\; </mo>}}\right)I_{jj}\left({\beta }_o^{\mathrm{<mo>\; *\; </mo>}}\right)+O_p\left(1\right)=0$

implying

$\sqrt{n}\sqrt{I_{jj}\left({\beta }_o^{\mathrm{<mo>\; *\; </mo>}}\right)}{Z}_j-n\left({\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_{o_j}^{\mathrm{<mo>\; *\; </mo>}}\right)I_{jj}\left({\beta }_o^{\mathrm{<mo>\; *\; </mo>}}\right)+O_p\left(1\right)=0.$ (51)

Rearranging (51) we have

$n{I}_{jj}\left({\beta }_o^{\mathrm{<mo>\; *\; </mo>}}\right)\left({\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_{o_j}^{\mathrm{<mo>\; *\; </mo>}}\right)+O_p\left(1\right)=\sqrt{n}\sqrt{I_{jj}\left({\beta }_o^{\mathrm{<mo>\; *\; </mo>}}\right)}{Z}_j$ (52)

for $j=\mathrm{1,2,}\cdots ,p$ where $Z_j~\left(\mathrm{0,}{V}_{{\beta }_j}\right)$ where $V_{{\beta }_j}$ is the variance of the j^th element of the covariance matrix $V_{\beta }$ of the distribution $=G^{{\rm T}}{N}_p\left(0\mathrm{,}V\right)$ and thus $Z_j$ is the standard normal distribution for the j^th element of ${\hat{\beta }}_n$ . Now dividing the left and right hand side of (52) by $\sqrt{n}\sqrt{I_{jj}\left({\beta }_o^{\mathrm{<mo>\; *\; </mo>}}\right)}$ we obtain

$\sqrt{n}\sqrt{I_{jj}\left({\beta }_o^{\mathrm{<mo>\; *\; </mo>}}\right)}\left({\hat{\beta }}_{n_j}^{\mathrm{<mo>\; *\; </mo>}}-{\beta }_{o_j}^{\mathrm{<mo>\; *\; </mo>}}\right)+O_p\left(1\right)=Z_j$ (53)

where $Z_j$ is the distribution of the j^th element of ${\hat{\beta }}_n$ and $Z~N_p\left(0\mathrm{,}{V}_{\beta }\right)$ . Using sequence (45), Equation (53) becomes

$k\sqrt{n}\sqrt{I_{jj}\left({\beta }_o^{\mathrm{<mo>\; *\; </mo>}}\right)}\left({\hat{\beta }}_{n_j}-{\beta }_{o_j}\right)+O_p\left(1\right)=Z_j$ (54)

for some ${\beta }_{o_j}^{\mathrm{<mo>\; *\; </mo>}}\to {\beta }_{o_j}$ and $j=\mathrm{1,2,}\cdots ,p$ where $Z~N_p\left(0\mathrm{,}{V}_{\beta }\right)$ and $V_{\beta }=G^{{\rm T}}VG$ . The distribution $Z_j$ is a normal distribution for each j^th element of the plug-in estimator ${\hat{\beta }}_n$ as described before in the analysis above.

Thus Equation (54) establishes the condition which implies local asymptotic normality (LAN) and differentiability in quadratic mean (DQM) for the estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ which implies that the rate of convergence is of order $\frac{1}{k\sqrt{n}}$ and rate

$k\sqrt{n}$ . This can also be achieved if we use the fact that the risk bound of the James-Stein shrinkage estimator is bounded by that of the MLE and the latter converges at the rate $\sqrt{n}$ . Hence the James-Stein shrinkage estimator is $k\sqrt{n}$ -consistent.

3.5. Simulation Plots

In this section the behaviour of the mean squared error (MSE) of the maximum likelihood estimator ${\hat{\theta }}_n$ is compared to that of the James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ as the sample size n increases. The statistical package R is used to simulate plots of the MSE for different sample size values of n using the R package library (MASS) to stimulate data which follow a multivariate normal distribution. The data is generated using a 3 × 3 correlation matrix ( $\rho$ ) to get the covariance matrix $\Sigma$ given by

$\Sigma =\left[\begin{array}{ccc}1& 0.3& 0.1\\ 0.3& 1& 0.2\\ 0.1& 0.2& 1\end{array}\right]$

which is symmetric and the variance in the major diagonal is 1 representing a standard normal variance. Thus we take $p=3$ and this meets the James-Stein classical condition of $p\ge 3$ . Since $X~N_3\left(\theta \mathrm{,}\Sigma \right)$ , we have

${\hat{\theta }}_n={\bar{X}}_n=\frac{1}{n}{\displaystyle \underset{i=1}{\overset{n}{\sum }}}{X}_{in}$ for $p=3$ (55)

making the MLE ${\hat{\theta }}_n$ a 3 × 1 matrix which implies that the dimension for the shrinkage estimator is also 3 × 1. Now knowing that the maximum likelihood estimator ${\hat{\theta }}_n$ is unbiased and the James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ is biased, the following expressions were used to calculate the mean squared error (MSE) of the two estimators. Using (55), we have

$MS{E}_{\theta }\left({\hat{\theta }}_n\right)={\Sigma }_n\left({\hat{\theta }}_n\right)={\Sigma }_n\left({\bar{X}}_n\right)=Var\left({\bar{X}}_n\right)$ (56)

as the mean squared error of ${\hat{\theta }}_n$ for $p=3$ since $b_{\theta }\left({\hat{\theta }}_n\right)=0$ . Similarly we have

$\begin{array}{c}MS{E}_{\theta }\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)={\Sigma }_n\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)+{\left[b_{\theta }\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)\right]}^2\\ =Var\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)+{\left[b_{\theta }\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)\right]}^2\end{array}$ (57)

for the James-Stein shrinkage estimator which becomes

$\begin{array}{c}MS{E}_{\theta }\left({\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}\right)={\Sigma }_n\left(k{\hat{\theta }}_n\right)+{\left[b_{\theta }\left(k{\hat{\theta }}_n\right)\right]}^2\\ =Var\left(k{\hat{\theta }}_n\right)+{\left[b_{\theta }\left(k{\hat{\theta }}_n\right)\right]}^2\end{array}$ (58)

where k is a shrinkage value which shrinks the maximum likelihood estimator ${\hat{\theta }}_n$ to a James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ for $p=3$ . Thus the mean squared error of the shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ in (58) is obtained by using (56). The shrinkage value k is evaluated using the expression

$k=\left(1-\frac{1}{{\bar{X}}_n^{{\rm T}}{\Sigma }_n\left({\bar{X}}_n\right){\bar{X}}_n}\right)=1-\frac{1}{{\bar{X}}_n^{{\rm T}}Var\left({\bar{X}}_n\right){\bar{X}}_n}$ (59)

for $p=3$ . The commands for all expressions and plots produced in R are provided in the appendix.

We present MSE plots obtained by simulating the mean squared error using the sample size values of n = 30, 2000 and 100,000. The MSE plots for both the James-Stein shrinkage estimator and maximum likelihood estimator for each sample size value considered are plotted on the same graph for easy comparison of the MSE trends. We begin by considering a small sample size value of n = 30 to compare the the way the MSE line plots change from one particular point to the other. Since we are interested in asymptotic behaviour, we increase the sample size value to 2000 and then 100,000 to analyse the MSE trends and the rate at which the line plots become smooth.

Figure 1. MSE plots for the MLE and JSSE for n = 30.

Figure 2. MSE for the JSSE and MLE for n = 2000.

Figure 3. MSE plots for JSSE and MLE for n = 10,000.

Collectively the scaled plots show that there is some reduction in the mean squared error of the James-Stein shrinkage estimator compared to that of the initial estimator (MLE). The trend in mean squared error for both the maximum likelihood estimator and the James-Stein shrinkage estimator shows that as the sample size value n increases, the MSE values converge to some value. The MSE plots suggest that the James-Stein shrinkage estimator converges to a lower MSE value 0.9 compared to the maximum likelihood estimator which converges to a MSE value 1.0. They also show that the James-Stein shrinkage estimator converges faster compared to the MLE though the difference is minimal.

4. Conclusions and Suggestions

In the study, we explored the asymptotic properties of the James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ which is obtained by shrinking a MLE ${\hat{\theta }}_n$ . Asymptotic consistency and efficiency of the shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ were investigated. From the regularity conditions, the MLE is known to be unbiased, consistent and efficient as the sample size value $n\to \infty$ . Therefore, the study analysed these asymptotic properties by checking whether the new estimator (shrinkage) obtained after shrinking possesses them. Thus the study examined whether the shrinking process has an effect on these properties. The results show that the James-Stein shrinkage estimator ${\hat{\beta }}_n^{\mathrm{<mo>\; *\; </mo>}}$ is asymptotically consistent and efficient. The study also showed that the shrinkage estimator (JSSE) is asymptotically biased, a property it possesses even for small values of the sample size value n. The bias is brought by the shrinking factor k given in Equation (59). We therefore see that the shrinking process introduces bias to the estimators obtained but it preserves asymptotic consistency and efficiency, and more importantly, it reduces the MSE.

Thus the James-Stein shrinkage estimator obtained by shrinking techniques proves to be useful though it is biased. This estimator is more effective than the maximum likelihood estimator as shown in this study and by Hansen [10] . The study also showed that the JSSE is stable for large values of the sample size value n, making it suitable in practical applications since large sample size values are normally considered for effective estimation. Since error is always there in estimation, we justify shrinking (minimising error) as a very important technique for yielding effective estimators.

The study has investigated the asymptotic behaviour of the James-Stein shrinkage estimator. Asymptotic properties analysed in the study include rate of convergence, consistency and efficiency. The results show that the James-Stein shrinkage estimator has a lower mean squared error compared to the maximum likelihood estimator though it is biased. The results further show that the JSSE is asymptotically consistent and efficient.

Acknowledgements

The authors gratefully acknowledge the Department of Mathematics and Statistics at the University of Zambia for supporting this work. Sincere thanks to the managing editor Alline Xiao for a rare attitude of high quality.

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Stein, C. (1956) Inadmissibility of the Usual Estimator for the Mean of a Multivariate Normal Distribution. In: Neyman, J., ed., Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, Volume I, Statistical Laboratory of the University of California, Berkeley, 197-206. https://doi.org/10.1525/9780520313880-018
[2]	James, W. and Stein, C. (1961) Estimation with Quadratic Loss. In: Neyman, J., ed., Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Statistical Laboratory of the University of California, Berkeley.
[3]	Baranchik, A.J. (1964) Multiple Regression and Estimation of the Mean of a Multivariate Normal Distribution. Technical Report No. 51. Department of Statistics, Stanford University, Stanford, CA.
[4]	Berger, J.O. (1976) Minimax Estimation of Multivariate Normal Mean with Arbitrary Quadratic Loss. Journal of Multivariate Analysis, 6, 256-264. https://doi.org/10.1016/0047-259X(76)90035-X
[5]	Stein, C. (1981) Estimation of the Mean of a Multivariate Normal Distribution. Annals of Statistics, 9, 1135-1151. https://doi.org/10.1214/aos/1176345632
[6]	Carter, R.L. and Ullah, A. (1984) The Sampling Distribution of Estimators and Their F-Ratios in Regression Model. Journal of Econometrics, 25, 109-122. https://doi.org/10.1016/0304-4076(84)90040-X
[7]	George, E. I. (1986) Minimax Multiple Shrinkage Estimation. Annals of Statistics, 14, 188-205. https://doi.org/10.1214/aos/1176349849
[8]	Geyer, C.J. (1994) On the Asymptotic of Constrained M-Estimation. Annals of Statistics, 22, 1993-2010. https://doi.org/10.1214/aos/1176325768
[9]	Hansen, E.B. (2008) Generalized Shrinkage Estimators. https://web-docs.stern.nyu.edu/old_web/emplibrary/shrink3.pdf
[10]	Hansen, E.B. (2016) Efficient Shrinkage in Parametric Models. Journals of Econometrics, 190, 188-205. https://doi.org/10.1016/j.jeconom.2015.09.003
[11]	Efron, B. (1975) Biased versus Unbiased Estimation. In Advances in Mathematics, Academic Press, New York. https://doi.org/10.1016/0001-8708(75)90114-0
[12]	Newey, W.K. and Mcfadden, D.L. (1994) Large Sample Estimation and Hypothesis Testing. University Press, Holland, 2113-2245. https://doi.org/10.1016/S1573-4412(05)80005-4
[13]	Stone, C.J. (1974) Asymptotic Properties of Estimators of a Location Parameter. The Annals of Statistics, 6, 1127-1137. https://doi.org/10.1214/aos/1176342869
[14]	Hogdes, J.L. and Lehman, E.L. (1975) Some Applications of the Cramér-Rao Inequality. In Proceedings of Second Berkeley Symposium on Mathematical Statistics and Probability, University of California Press, Berkeley, CA, 13-22.
[15]	Hoeffding, W. (1948) A Class of Statistics with Asymptotically Normal Distribution. Annals of Mathematical Statistics, 19, 293-325. https://doi.org/10.1214/aoms/1177730196