Mean squared prediction error

In statistics the mean squared prediction error of a smoothing or curve fitting procedure is the expected value of the squared difference between the fitted values implied by the predictive function ${\widehat {g}}$ and the values of the (unobservable) function g. It is an inverse measure of the explanatory power of ${\widehat {g}},$ and can be used in the process of cross-validation of an estimated model.

If the smoothing or fitting procedure has operator matrix (i.e., hat matrix) L, which maps the observed values vector $y$ to predicted values vector ${\hat {y}}$ via ${\hat {y}}=Ly,$ then

$\operatorname {MSPE} (L)=\operatorname {E} \left[\left(g(x_{i})-{\widehat {g}}(x_{i})\right)^{2}\right].$

The MSPE can be decomposed into two terms (just like mean squared error is decomposed into bias and variance); however, for MSPE one term is the sum of squared biases of the fitted values and the other is the sum of variances of the fitted values:

$\operatorname {MSPE} (L)=\sum _{i=1}^{n}\left(\operatorname {E} \left[{\widehat {g}}(x_{i})\right]-g(x_{i})\right)^{2}+\sum _{i=1}^{n}\operatorname {var} \left[{\widehat {g}}(x_{i})\right].$

Knowledge of g is required in order to calculate MSPE exactly.

Estimation of MSPE

For the model $y_{i}=g(x_{i})+\sigma \varepsilon _{i}$ where $\varepsilon _{i}\sim {\mathcal {N}}(0,1)$, one may write

$\operatorname {MSPE} (L)=g'(I-L)'(I-L)g+\sigma ^{2}\operatorname {tr} \left[L'L\right].$

The first term is equivalent to

$\sum _{i=1}^{n}\left(\operatorname {E} \left[{\widehat {g}}(x_{i})\right]-g(x_{i})\right)^{2}=\operatorname {E} \left[\sum _{i=1}^{n}\left(y_{i}-{\widehat {g}}(x_{i})\right)^{2}\right]-\sigma ^{2}\operatorname {tr} \left[\left(I-L\right)'\left(I-L\right)\right].$

Thus,

$\operatorname {MSPE} (L)=\operatorname {E} \left[\sum _{i=1}^{n}\left(y_{i}-{\widehat {g}}(x_{i})\right)^{2}\right]-\sigma ^{2}\left(n-2\operatorname {tr} \left[L\right]\right).$

If $\sigma ^{2}$ is known or well-estimated by ${\widehat {\sigma }}^{2}$, it becomes possible to estimate MSPE by

$\operatorname {\widehat {MSPE}} (L)=\sum _{i=1}^{n}\left(y_{i}-{\widehat {g}}(x_{i})\right)^{2}-{\widehat {\sigma }}^{2}\left(n-2\operatorname {tr} \left[L\right]\right).$

Colin Mallows advocated this method in the construction of his model selection statistic C_p, which is a normalized version of the estimated MSPE:

$C_{p}={\frac {\sum _{i=1}^{n}\left(y_{i}-{\widehat {g}}(x_{i})\right)^{2}}{{\widehat {\sigma }}^{2}}}-n+2\operatorname {tr} \left[L\right].$

where p comes from the fact that the number of parameters p estimated for a parametric smoother is given by $p=\operatorname {tr} \left[L\right]$, and C is in honor of Cuthbert Daniel.

See also

• Mean squared error
• Errors and residuals in statistics
• Law of total variance