Minimum-variance unbiased estimator

In statistics a uniformly minimum-variance unbiased estimate or minimum-variance unbiased estimate (UMVUE or MVUE) is an unbiased estimate that has lower variance than any other unbiased estimate for all possible values of the parameter.

The question of determining the UMVUE, if one exists, for a particular problem is important for practical statistics, since less-than-optimal procedures would naturally be avoided, other things being equal. This has led to substantial development of statistical theory related to the problem of optimal estimation. While the particular specification of "optimal" here — requiring unbiasedness and measuring "goodness" using the variance — may not always be what is wanted for any given practical situation, it is one where useful and generally applicable results can be found.

Definition

Consider estimation of $g(\theta )$ based on data $X_{1},X_{2},\ldots ,X_{n}$ i.i.d. from some member of a family of densities $p_{\theta },\theta \in \Omega$ , where $\Omega$ is the parameter space. An unbiased estimate $\delta (X_{1},X_{2},\ldots ,X_{n})$ of $g(\theta )$ is UMVU if $\forall \theta \in \Omega$ ,

\mathrm {var} (\delta (X_{1},X_{2},\ldots ,X_{n}))\leq \mathrm {var} ({\tilde {\delta }}(X_{1},X_{2},\ldots ,X_{n}))

for any other unbiased estimate ${\tilde {\delta }}.$

If an unbiased estimate of $g(\theta )$ exists, then one can prove there is an essentially unique MVUE. Using the Rao–Blackwell theorem one can also prove that determining the MVUE is simply a matter of finding a complete sufficient statistic for the family $p_{\theta },\theta \in \Omega$ and conditioning any unbiased estimate on it.

Further, by the Lehmann–Scheffé theorem, an unbiased estimate that is a function of a complete, sufficient statistic is the UMVU estimate.

Put formally, suppose $\delta (X_{1},X_{2},\ldots ,X_{n})$ is unbiased for $g(\theta )$ , and that $T$ is a complete sufficient statistic for the family of densities. Then

\eta (X_{1},X_{2},\ldots ,X_{n})=\mathrm {E} (\delta (X_{1},X_{2},\ldots ,X_{n})|T)\,

is the MVUE for $g(\theta ).$

A Bayesian analog is a Bayes estimate, particularly with minimum mean square error (MMSE).

Estimate selection

An efficient estimate need not exist, but if it does and if it unbiased, it is the MVUE. Since the mean squared error (MSE) of an estimate δ is

\operatorname {MSE} (\delta )=\mathrm {var} (\delta )+[\mathrm {bias} (\delta )]^{2}\

the MVUE minimizes MSE among unbiased estimates. In some cases biased estimates have lower MSE because they have a smaller variance than does any unbiased estimate; see estimate bias.

Example

Consider the data to be a single observation from an absolutely continuous distribution on $\mathbb {R}$ with density

p_{\theta }(x)={\frac {\theta e^{-x}}{(1+e^{-x})^{\theta +1}}}

and we wish to find the UMVU estimate of

g(\theta )={\frac {1}{\theta ^{2}}}

First we recognize that the density can be written as

{\frac {e^{-x}}{1+e^{-x}}}\exp(-\theta \log(1+e^{-x})+\log(\theta ))

Which is an exponential family with sufficient statistic $T=\mathrm {log} (1+e^{-x})$ . In fact this is a full rank exponential family, and therefore $T$ is complete sufficient. See exponential family for a derivation which shows

\mathrm {E} (T)={\frac {1}{\theta }},\quad \mathrm {var} (T)={\frac {1}{\theta ^{2}}}

Therefore

\mathrm {E} (T^{2})={\frac {2}{\theta ^{2}}}

Clearly $\delta (X)={\frac {T^{2}}{2}}$ is unbiased, thus the UMVU estimate is

\eta (X)=\mathrm {E} (\delta (X)|T)=\mathrm {E} \left(\left.{\frac {T^{2}}{2}}\,\right|\,T\right)={\frac {T^{2}}{2}}={\frac {\log(1+e^{-X})^{2}}{2}}

This example illustrates that an unbiased function of the complete sufficient statistic will be UMVU.

Other examples

For a normal distribution with unknown mean and variance, the sample mean and (unbiased) sample variance are the MVUEs for the population mean and population variance.
However, the sample standard deviation is not unbiased for the population standard deviation – see unbiased estimation of standard deviation.

Further, for other distributions the sample mean and sample variance are not in general MVUEs – for a uniform distribution with unknown upper and lower bounds, the mid-range is the MVUE for the population mean.
If k exemplars are chosen (without replacement) from a discrete uniform distribution over the set {1, 2, ..., N} with unknown upper bound N, the MVUE for N is

{\frac {k+1}{k}}m-1,

where m is the sample maximum. This is a scaled and shifted (so unbiased) transform of the sample maximum, which is a sufficient and complete statistic. See German tank problem for details.

References

Keener, Robert W. (2006). Statistical Theory: Notes for a Course in Theoretical Statistics. Springer. pp. 47–48, 57–58.