Minimizing the Variance of a Weighted Average

Doron J. Shahar

Department of Mathematics, University of Arizona, Tucson, Arizona, USA.

DOI: 10.4236/ojs.2017.72017                

Abstract

It is common practice in science to take a weighted average of estimators of a single parameter. If the original estimators are unbiased, any weighted average will be an unbiased estimator as well. The best estimator among the weighted averages can be obtained by choosing weights that minimize the variance of the weighted average. If the variances of the individual estimators are given, the ideal weights have long been known to be the inverse of the variance. Nonetheless, I have not found a formal proof of this result in the literature. In this article, I provide three different proofs of the ideal weights.

Keywords

Variance, Weighted Average, Minimization

1. Introduction

Oftentimes in science, multiple point estimators of the same parameter are com- bined to form a better estimator. One method of forming the new estimator is taking a weighted average of the original estimators. If the original estimators are unbiased, the weighted average is guaranteed to be unbiased as well.

A weighted average may be used to combine the results of several studies (meta- analysis), or when several estimates are obtained within a study. For example, to deconfound it may be necessary to stratify on a covariate when estimating an effect. Assuming that the effect is the same in every stratum of the covariate, we may take a weighted average of the stratum-specific estimates.

The question remains though as to which weights should be chosen. Since the estimator will be unbiased regardless of the weights, we only need to consider the variance. In particular, the weights should be chosen to minimize the va- riance of the weighted average. Although it has long been known that the ideal weights should be the inverse of the variance, I have not found any complete, formal proof in the literature. Several sources mention the ideal weights either in general or for specific cases without any proof [1] - [6] . Others mention something similar to the ideal weights but again without proof [7] [8] . Hedges offers a so- called proof in his 1981 paper [9] , which is far from a complete proof. The first two sentences of his proof contain the same content as the first two sentences and the last sentence of proof 1 in this paper. Hedges then continues to prove approximations for the ideal weights under a certain condition. In his 1982 and 1983 papers, he writes that this result is “easy to show” referencing his 1981 (and 1982) papers [10] [11] . Goldberg and Kercheval [12] provide a “proof” that contains only slightly more content than Hedges’ proof in that they mention the use of Lagrange multipliers. Proof 1 in this paper goes over the details tho- roughly. Cochran also mentions the ideal weights, but proves only that these weights give the maximum likelihood estimate when the estimators are inde- pendent and normally distributed about a common mean [13] . Lastly, the pro- blem of finding the ideal weights is included as an exercise (exercise 7.42) in Casella and Berger [14] . The problem, however, is not worked out in their solution manual [15] . A version of the problem when taking a weighted average of only two estimators is also an exercise (exercise 24.1) in Anderson and Ban- croft [1] .

Searching through articles dating back to the 1930s, it seems that this basic result has not been formally proven in the literature. One reason may be that the result depends on the variances of the estimators being known. The case when the variances are unknown is more difficult and attracted more attention. For example, a few articles briefly mentioned the ideal weights when the variances are known before continuing to discuss the case when the variances are unknown [2] [4] [5] [13] . In this paper, I present three proofs of the ideal weights that minimize the variance of a weighted average.

2. Three Proofs

Let $X_1,\cdots ,X_n\left(n\ge 2\right)$ be estimators of a single parameter $\theta$ . In practice, the $X_i$ are independent, and they are often assumed to be unbiased. If that’s the case, then any weighted average $X={\displaystyle {\sum }_{i=1}^n}{w}_i{X}_i$ ( ${\displaystyle {\sum }_{i=1}^n}{w}_i=1$ and $w_i\ge 0$ for all $i$ ) is an unbiased estimator of $\theta$ . Since the estimator is unbiased regardless of the weights, we want to choose weights that minimize the variance of $X$ .

As far as the ideal weights are concerned, it is not necessary though that the $X_i$ be independent and unbiased. The proof of the ideal weights only requires that the $X_i$ be uncorrelated and have a finite non-zero variance.

Proposition 1. If $X_1\mathrm{,}\cdots \mathrm{,}{X}_n\left(n\ge 2\right)$ are uncorrelated random variables with finite non-zero variances, then $\text{Var}\left({\displaystyle {\sum }_{i=1}^n}{w}_i{X}_i\right)$ is minimized when

$w_i=\frac{\frac{1}{\text{Var}\left(X_i\right)}}{{\displaystyle {\sum }_{j=1}^n\frac{1}{\text{Var}\left(X_j\right)}}}$

and its minimum value is

$\frac{1}{{\displaystyle {\sum }_{i=1}^n\frac{1}{\text{Var}\left(X_i\right)}}}$

The first proof uses the method of Lagrange multipliers.

Proof 1: $\text{Var}\left({\displaystyle {\sum }_{i=1}^n}{w}_i{X}_i\right)={\displaystyle {\sum }_{i=1}^n}\text{Var}\left(w_i{X}_i\right)={\displaystyle {\sum }_{i=1}^n}{w}_i^2\text{Var}\left(X_i\right)$ , because the $X_i$ are uncorrelated. We wish to minimize the previous expression under the con- straint that ${\displaystyle {\sum }_{i=1}^n}{w}_i=1$ and $w_i\ge 0$ for all $i$ . The set $T$ of all $\left(w_1\mathrm{,}\cdots \mathrm{,}{w}_n\right)\in {ℝ}^n$ for which ${\displaystyle {\sum }_{i=1}^n}{w}_i=1$ and $w_i\ge 0$ for all $i$ is closed and bounded. The extrema in the interior of $T$ can be found by considering only the first con- straint, which may be written as ${\displaystyle {\sum }_{i=1}^n}{w}_i-1=0$ . Later we shall find the extrema on the boundary of $T$ . To find the extrema in the interior of $T$ , let

$F\left(w_1,\cdots ,w_n,\lambda \right)={\displaystyle \underset{i=1}{\overset{n}{\sum }}}{w}_i^2\text{Var}\left(X_i\right)-\lambda \left({\displaystyle \underset{i=1}{\overset{n}{\sum }}}{w}_i-1\right)$ (1)

By the method of Lagrange multipliers, the values of $w_1\mathrm{,}\cdots \mathrm{,}{w}_n$ for which

$\partial F/\partial w_j=0$ are the critical points of $\text{Var}\left({\displaystyle {\sum }_{i=1}^n}{w}_i{X}_i\right)$ . (These contain all the ex-

trema of $\text{Var}\left({\displaystyle {\sum }_{i=1}^n}{w}_i{X}_i\right)$ in the interior of $T$ .) $\partial F/\partial w_j=2w_j\text{Var}\left(X_j\right)-\lambda$ equals zero only when $w_j=\left(\lambda /2\right)\text{/Var}\left(X_j\right)$ .

$1={\displaystyle \underset{j=1}{\overset{n}{\sum }}}{w}_j={\displaystyle \underset{j=1}{\overset{n}{\sum }}}\frac{\lambda /2}{\text{Var}\left(X_j\right)}=\frac{\lambda }{2}{\displaystyle \underset{j=1}{\overset{n}{\sum }}}\frac{1}{\text{Var}\left(X_j\right)}$ (2)

Therefore,

$\lambda =\frac{2}{{\displaystyle {\sum }_{j=1}^n\frac{1}{\text{Var}\left(X_j\right)}}}$ (3)

and

$w_i=\frac{\lambda /2}{\text{Var}\left(X_i\right)}=\frac{\frac{1}{\text{Var}\left(X_i\right)}}{{\displaystyle {\sum }_{j=1}^n\frac{1}{\text{Var}\left(X_j\right)}}}>0$ (4)

$\left(w_1\mathrm{,}\cdots \mathrm{,}{w}_n\right)$ is indeed in the interior of $T$ since $w_i>0$ . For these values of $w_i$ ,

${\displaystyle \underset{i=1}{\overset{n}{\sum }}}{w}_i^2\text{Var}\left(X_i\right)={\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\left(\frac{\frac{1}{\text{Var}\left(X_i\right)}}{{\displaystyle {\sum }_{j=1}^n\frac{1}{\text{Var}\left(X_j\right)}}}\right)}^2\text{Var}\left(X_i\right)=\frac{{\displaystyle {\sum }_{i=1}^n\frac{1}{\text{Var}\left(X_i\right)}}}{{\left({\displaystyle {\sum }_{j=1}^n\frac{1}{\text{Var}\left(X_j\right)}}\right)}^2}=\frac{1}{{\displaystyle {\sum }_{i=1}^n\frac{1}{\text{Var}\left(X_i\right)}}}$ (5)

Next, let’s find the extrema on the boundary of $T$ . The boundary of $T$ is characterized by having some of the $w_i$ equal zero. For any point on the boun- dary, let $S=\left\{i:w_i\ne 0\right\}$ . At such a point, $\text{Var}\left({\displaystyle {\sum }_{i=1}^n}{w}_i{X}_i\right)=\text{Var}\left({\displaystyle {\sum }_{i\in S}}{w}_i{X}_i\right)$ . Using the method of Lagrange multipliers again, the critical points of $\text{Var}\left({\displaystyle {\sum }_{i=1}^n}{w}_i{X}_i\right)=\text{Var}\left({\displaystyle {\sum }_{i\in S}}{w}_i{X}_i\right)$ are found to be $\left(w_1\mathrm{,}\cdots \mathrm{,}{w}_n\right)$ where

$w_i=\{\begin{array}{ll}\frac{\frac{1}{\text{Var}\left(X_i\right)}}{{\displaystyle {\sum }_{j\in S}\frac{1}{\text{Var}\left(X_j\right)}}}\hfill & \text{if}i\in S\hfill \\ 0\hfill & \text{if}i\notin S\hfill \end{array}$ (6)

(These contain all the extrema of $\text{Var}\left({\displaystyle {\sum }_{i=1}^n}{w}_i{X}_i\right)$ on the boundary of $T$ .) For these values of $w_i$ ,

${\displaystyle \underset{i=1}{\overset{n}{\sum }}}{w}_i^2\text{Var}\left(X_i\right)={\displaystyle \underset{i\in S}{\sum }}{\left(\frac{\frac{1}{\text{Var}\left(X_i\right)}}{{\displaystyle {\sum }_{j\in S}\frac{1}{\text{Var}\left(X_j\right)}}}\right)}^2\text{Var}\left(X_i\right)=\frac{{\displaystyle {\sum }_{i\in S}\frac{1}{\text{Var}\left(X_i\right)}}}{{\left({\displaystyle {\sum }_{j\in S}\frac{1}{\text{Var}\left(X_j\right)}}\right)}^2}=\frac{1}{{\displaystyle {\sum }_{i\in S}\frac{1}{\text{Var}\left(X_i\right)}}}$ (7)

Note that

$\frac{1}{{\displaystyle {\sum }_{i\in S}\frac{1}{\text{Var}\left(X_i\right)}}}\ge \frac{1}{{\displaystyle {\sum }_{i=1}^n\frac{1}{\text{Var}\left(X_i\right)}}}$ (8)

That is, of all critical points, the one in the interior of $T$ minimizes

${\displaystyle {\sum }_{i=1}^n}{w}_i^2\text{Var}\left(X_i\right)$ . Therefore, $\text{Var}\left({\displaystyle {\sum }_{i=1}^n}{w}_i{X}_i\right)={\displaystyle {\sum }_{i=1}^n}{w}_i^2\text{Var}\left(X_i\right)$ is minimized

when

$w_i=\frac{\frac{1}{\text{Var}\left(X_i\right)}}{{\displaystyle {\sum }_{j=1}^n\frac{1}{\text{Var}\left(X_j\right)}}}$

and its minimum value is

$\frac{1}{{\displaystyle {\sum }_{i=1}^n\frac{1}{\text{Var}\left(X_i\right)}}}$

The second proof is done by induction.

Proof 2: The case $n=2$ will be our base case for the induction.

$\begin{array}{c}\text{Var}\left(w_1{X}_1+w_2{X}_2\right)=w_1^2\text{Var}\left(X_1\right)+w_2^2\text{Var}\left(X_2\right)\\ =w_1^2\text{Var}\left(X_1\right)+{\left(1-w_1\right)}^2\text{Var}\left(X_2\right)\\ =w_1^2\left(\text{Var}\left(X_1\right)+\text{Var}\left(X_2\right)\right)-2w_1\text{Var}\left(X_2\right)+\text{Var}\left(X_2\right)\end{array}$ (9)

The global minimum of the above expression occurs when

$w_1=\frac{\text{Var}\left(X_2\right)}{\text{Var}\left(X_1\right)+\text{Var}\left(X_2\right)}=\frac{\frac{1}{\text{Var}\left(X_1\right)}}{\frac{1}{\text{Var}\left(X_1\right)}+\frac{1}{\text{Var}\left(X_2\right)}}$ (10)

and

$w_2=1-w_1=\frac{\frac{1}{\text{Var}\left(X_2\right)}}{\frac{1}{\text{Var}\left(X_1\right)}+\frac{1}{\text{Var}\left(X_2\right)}}$ (11)

The minimum variance is then

${\displaystyle \underset{i=1}{\overset{2}{\sum }}}{\left(\frac{\frac{1}{\text{Var}\left(X_i\right)}}{{\displaystyle {\sum }_{j=1}^2\frac{1}{\text{Var}\left(X_j\right)}}}\right)}^2\text{Var}\left(X_i\right)=\frac{{\displaystyle {\sum }_{i=1}^2\frac{1}{\text{Var}\left(X_i\right)}}}{{\left({\displaystyle {\sum }_{j=1}^2\frac{1}{\text{Var}\left(X_j\right)}}\right)}^2}\mathrm{<mo>\; =\; </mo>}\frac{1}{{\displaystyle {\sum }_{i=1}^2\frac{1}{\text{Var}\left(X_i\right)}}}$ (12)

For the induction step, suppose that for some $n\ge 2$ , $\text{Var}\left({\displaystyle {\sum }_{i=1}^n}{w}_i{X}_i\right)$ is minimized when

$w_i=\frac{\frac{1}{\text{Var}\left(X_i\right)}}{{\displaystyle {\sum }_{j=1}^n\frac{1}{\text{Var}\left(X_j\right)}}}$

and its minimum value is

$\frac{1}{{\displaystyle {\sum }_{i=1}^n\frac{1}{\text{Var}\left(X_i\right)}}}$

Then,

$\begin{array}{c}\text{Var}\left({\displaystyle \underset{i=1}{\overset{n+1}{\sum }}}{w}_i{X}_i\right)=\text{Var}\left(\left({\displaystyle \underset{j=1}{\overset{n}{\sum }}}{w}_j\right)\left({\displaystyle \underset{i=1}{\overset{n}{\sum }}}\frac{w_i}{{\displaystyle {\sum }_{j=1}^n{w}_j}}{X}_i\right)+w_{n+1}{X}_{n+1}\right)\\ ={\left({\displaystyle \underset{j=1}{\overset{n}{\sum }}}{w}_j\right)}^2\text{Var}\left({\displaystyle \underset{i=1}{\overset{n}{\sum }}}\frac{w_i}{{\displaystyle {\sum }_{j=1}^n{w}_j}}{X}_i\right)+w_{n+1}^2\text{Var}\left(X_{n+1}\right)\\ ={\left(1-w_{n+1}\right)}^2\text{Var}\left({\displaystyle \underset{i=1}{\overset{n}{\sum }}}{u}_i{X}_i\right)+w_{n+1}^2\text{Var}\left(X_{n+1}\right)\\ =w_{n+1}^2\left(\text{Var}\left(X_{n+1}\right)+\text{Var}\left({\displaystyle \underset{i=1}{\overset{n}{\sum }}}{u}_i{X}_i\right)\right)-2w_{n+1}\text{Var}\left({\displaystyle \underset{i=1}{\overset{n}{\sum }}}{u}_i{X}_i\right)+\text{Var}\left({\displaystyle \underset{i=1}{\overset{n}{\sum }}}{u}_i{X}_i\right)\end{array}$ (13)

where $u_i=w_i/{\displaystyle {\sum }_{j=1}^n}{w}_j$ are weights that do not depend on $w_{n+1}$ . So for any possible values of $u_i$ , the above expression is minimized when

$w_{n+1}=\frac{\text{Var}\left({\displaystyle {\sum }_{i=1}^n{u}_i{X}_i}\right)}{\text{Var}\left(X_{n+1}\right)+\text{Var}\left({\displaystyle {\sum }_{i=1}^n{u}_i{X}_i}\right)}$ (14)

By plugging the above value for $w_{n+1}$ into Equation (13), we find a lower bound for the variance of the weighted average:

$\text{Var}\left({\displaystyle \underset{i=1}{\overset{n+1}{\sum }}}{w}_i{X}_i\right)\ge \frac{\text{Var}\left(X_{n+1}\right)\text{Var}\left({\displaystyle {\sum }_{i=1}^n{u}_i{X}_i}\right)}{\text{Var}\left(X_{n+1}\right)+\text{Var}\left({\displaystyle {\sum }_{i=1}^n{u}_i{X}_i}\right)}$ (15)

where equality is achieved for the specified value of $w_{n+1}$ .

The variance of the weighted average will be minimized when it equals the right side of the above inequality and the right side of the inequality is minimized. The right side of the inequality is minimized when $\text{Var}\left({\displaystyle {\sum }_{i=1}^n}{u}_i{X}_i\right)$ is minimized. We have assumed in the induction step that $\text{Var}\left({\displaystyle {\sum }_{i=1}^n}{u}_i{X}_i\right)$ is minimized when

$u_i=\frac{\frac{1}{\text{Var}\left(X_i\right)}}{{\displaystyle {\sum }_{j=1}^n\frac{1}{\text{Var}\left(X_j\right)}}}$

and its minimum value is

$\frac{1}{{\displaystyle {\sum }_{i=1}^n\frac{1}{\text{Var}\left(X_i\right)}}}$

Therefore, $\text{Var}\left({\displaystyle {\sum }_{i=1}^{n+1}}{w}_i{X}_i\right)$ is minimized when

$w_{n+1}=\frac{\text{Var}\left({\displaystyle {\sum }_{i=1}^n{u}_i{X}_i}\right)}{\text{Var}\left(X_{n+1}\right)+\text{Var}\left({\displaystyle {\sum }_{i=1}^n{u}_i{X}_i}\right)}=\frac{\frac{1}{{\displaystyle {\sum }_{j=1}^n\frac{1}{\text{Var}\left(X_j\right)}}}}{\text{Var}\left(X_{n+1}\right)+\frac{1}{{\displaystyle {\sum }_{j=1}^n\frac{1}{\text{Var}\left(X_j\right)}}}}=\frac{\frac{1}{\text{Var}\left(X_{n+1}\right)}}{{\displaystyle {\sum }_{j=1}^{n+1}\frac{1}{\text{Var}\left(X_j\right)}}}$ (16)

and

$w_i=\left({\displaystyle \underset{j=1}{\overset{n}{\sum }}}{w}_j\right)u_i=\left(1-w_{n+1}\right)u_i=\left(1-\frac{\frac{1}{\text{Var}\left(X_{n+1}\right)}}{{\displaystyle {\sum }_{j=1}^{n+1}\frac{1}{\text{Var}\left(X_j\right)}}}\right)\frac{\frac{1}{\text{Var}\left(X_i\right)}}{{\displaystyle {\sum }_{j=1}^n\frac{1}{\text{Var}\left(X_j\right)}}}=\frac{\frac{1}{\text{Var}\left(X_i\right)}}{{\displaystyle {\sum }_{j=1}^{n+1}\frac{1}{\text{Var}\left(X_j\right)}}}$ (17)

for $i\in \left\{\mathrm{1,}\cdots \mathrm{,}n\right\}.$

Therefore, $\text{Var}\left({\displaystyle {\sum }_{i=1}^{n+1}}{w}_i{X}_i\right)$ is minimized when

$w_i=\frac{\frac{1}{\text{Var}\left(X_i\right)}}{{\displaystyle {\sum }_{j=1}^{n+1}\frac{1}{\text{Var}\left(X_j\right)}}}$

for all $i$ , and its minimum value is

$\frac{1}{{\displaystyle {\sum }_{i=1}^{n+1}\frac{1}{\text{Var}\left(X_i\right)}}}$

This completes the induction step, and the proof.

The third proof utilizes the Cauchy-Schwarz inequality.

Proof 3: Using the Cauchy-Schwarz inequality, we obtain

$\begin{array}{c}1={\left|{\displaystyle \underset{i=1}{\overset{n}{\sum }}}{w}_i\right|}^2={\left|{\displaystyle \underset{i=1}{\overset{n}{\sum }}}\left(w_i\sqrt{\text{Var}\left(X_i\right)}\right)\left(\frac{1}{\sqrt{\text{Var}\left(X_i\right)}}\right)\right|}^2\\ \le \left({\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\left(w_i\sqrt{\text{Var}\left(X_i\right)}\right)}^2\right)\cdot \left({\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\left(\frac{1}{\sqrt{\text{Var}\left(X_i\right)}}\right)}^2\right)\\ =\left({\displaystyle \underset{i=1}{\overset{n}{\sum }}}{w}_i^2\text{Var}\left(X_i\right)\right)\left({\displaystyle \underset{i=1}{\overset{n}{\sum }}}\frac{1}{\text{Var}\left(X_i\right)}\right)\end{array}$ (18)

Dividing both sides of Equation (18) by ${\displaystyle {\sum }_{i=1}^n}\frac{1}{\text{Var}\left(X_i\right)}$ , we have a lower bound

for $\text{Var}\left({\displaystyle {\sum }_{i=1}^n}{w}_i{X}_i\right)$ .

$\frac{1}{{\displaystyle {\sum }_{i=1}^n\frac{1}{\text{Var}\left(X_i\right)}}}\le {\displaystyle \underset{i=1}{\overset{n}{\sum }}}{w}_i^2\text{Var}\left(X_i\right)=\text{Var}\left({\displaystyle \underset{i=1}{\overset{n}{\sum }}}{w}_i{X}_i\right)$ (19)

By the Cauchy-Schwarz inequality, $\text{Var}\left({\displaystyle {\sum }_{i=1}^n}{w}_i{X}_i\right)$ equals the lower bound iff

$\left(w_1\sqrt{\text{Var}\left(X_1\right)}\mathrm{,}\cdots \mathrm{,}{w}_n\sqrt{\text{Var}\left(X_n\right)}\right)$

and

$\left(\frac{1}{\sqrt{\text{Var}\left(X_1\right)}}\mathrm{,}\cdots \mathrm{,}\frac{1}{\sqrt{\text{Var}\left(X_n\right)}}\right)$

are linearly dependent vectors. Since neither of these vectors is the zero vector, they are linearly dependent iff there exists an $\alpha$ such that $w_i\sqrt{\text{Var}\left(X_i\right)}=\alpha /\sqrt{\text{Var}\left(X_i\right)}$ for all $i$ . Therefore, $\text{Var}\left({\displaystyle {\sum }_{i=1}^n}{w}_i{X}_i\right)$ equals the lower bound iff there exists an $\alpha$ such that $w_i=\alpha /\text{Var}\left(X_i\right)$ for all $i$ . Since the $w_i$ are weights, this requires that

$\alpha =\frac{1}{{\displaystyle {\sum }_{j=1}^n\frac{1}{\text{Var}\left(X_j\right)}}}$ (20)

and

$w_i=\frac{\frac{1}{\text{Var}\left(X_i\right)}}{{\displaystyle {\sum }_{j=1}^n\frac{1}{\text{Var}\left(X_j\right)}}}.$ (21)

Therefore, $\text{Var}\left({\displaystyle {\sum }_{i=1}^n}{w}_i{X}_i\right)$ obtains the lower bound, and hence, is mini- mized when

$w_i=\frac{\frac{1}{\text{Var}\left(X_i\right)}}{{\displaystyle {\sum }_{j=1}^n\frac{1}{\text{Var}\left(X_j\right)}}}$

and its minimum value is

$\frac{1}{{\displaystyle {\sum }_{i=1}^n\frac{1}{\text{Var}\left(X_i\right)}}}$

3. Discussion

Given the frequent use of inverse variance weighting, it is surprising that the proof of proposition 1 was never published, to the best of my knowledge, in any book or journal. That the proofs use standard techniques is no excuse for their absence in the literature; there is value in a proof beyond the result it proves. For example, it is interesting that the proposition can be proven by induction and more succinctly using the Cauchy-Schwarz inequality.

Even more surprising are the trails of citations leading nowhere. It appears that generations of statisticians simply assumed that a proof has been published somewhere, relying on old, inaccurate citations. In that sense, this article not only offers three different proofs but also a broader lesson: every so often it is worth- while to review the history of well-known facts. Surprises are possible.

Acknowledgements

Many thanks to Eyal Shahar for first asking me to prove this result after he was unable to find the proof in the literature and for inspiring the second proof. (An earlier version of the first two proofs was posted on his university website in an appendix to his commentary on standardization.) I would also like to thank Sunder Sethuraman and Shankar Venkataramani for inspiring the third proof. And I would like to thank all three for having commented on a draft manuscript.

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Anderson, R.L. and Bancroft, T.A. (1952) Statistical Theory in Research. McGraw-Hill, New York, 358-366.
[2]	Meier, P. (1953) Variance of a Weighted Mean. Biometrics, 9, 59-73. https://doi.org/10.2307/3001633
[3]	Cochran, W.G. and Cox, G. (1957) Experimental Designs. John Wiley, New York, 561-562.
[4]	Graybill, F.A. and Deal, R.D. (1959) Combining Unbiased Estimators. Biometrics, 15, 543-550. https://doi.org/10.2307/2527652
[5]	Rukhin, A.L. (2007) Conservative Confidence Intervals Based on Weighted Mean Statistics. Statistics and Probability Letters, 77, 853-861.
[6]	Hartung, J., Knapp, G. and Sinha, B.K. (2008) Statistical Meta-Analysis with Applications. John Wiley & Sons, Hoboken, 44. https://doi.org/10.1002/9780470386347
[7]	Kempthorne, O. (1952) Design and Analysis of Experiments. John Wiley & Sons, New York, 534.
[8]	Cochran, W.G. (1954) The Combination of Estimates from Different Experiments. Biometrics, 10, 101-129. https://doi.org/10.2307/3001666
[9]	Hedges, L.V. (1981) Distribution Theory for Glass’s Estimator of Effect Size and Related Estimators. Journal of Educational Statistics, 6, 107-128. https://doi.org/10.2307/1164588
[10]	Hedges, L.V. (1982) Estimation of Effect Size from a Series of Independent Experiments. Psychological Bulletin, 92, 490-499. https://doi.org/10.1037/0033-2909.92.2.490
[11]	Hedges, L.V. (1983) A Random Effects Model for Effect Sizes. Psychological Bulletin, 93, 388-395. https://doi.org/10.1037/0033-2909.93.2.388
[12]	Goldberg, L.R. and Kercheval, A.N. (2002) t-Statistics for Weighted Means with Application to Risk Factor Models. The Journal of Portfolio Management, 28, 2.
[13]	Cochran, W.G. (1937) Problems Arising in the Analysis of a Series of Similar Experiments. Supplement to the Journal of the Royal Statistical Society, 4, 102-118. https://doi.org/10.2307/2984123
[14]	Casella, G. and Berger, R. (2001) Statistical Inference. 2nd Edition, Duxbury Press, Pacific Grove, 363.
[15]	Casella, G., Berger, R. and Sanatana, D. (2001) Solutions Manual for Statistical Inference, Second Edition. http://exampleproblems.com/Solutions-Casella-Berger.pdf