Statistical model validation

In statistics, model validation is the task of confirming that the outputs of a statistical model are acceptable with respect to the real data-generating process. In other words, model validation is the task of confirming that the outputs of a statistical model have enough fidelity to the outputs of the data-generating process that the objectives of the investigation can be satisfied.

Model validation can be based on two types of data: data that was used in the construction of the model and data that was not used in the construction. Validation based on the first type usually involves analyzing the goodness of fit of the model or analyzing whether the residuals seem to be random (i.e. residual diagnostics). Validation based on the second type usually involves analyzing whether the model's predictive performance deteriorates non-negligibly when applied to some new data.

For some classes of statistical models, specialized methods of performing validation are available. For example, if the statistical model was obtained via a regression, then specialized analyses for regression validation exist and are generally employed.

When doing a validation, there are three notable causes of potential difficulty, according to the Encyclopedia of Statistical Sciences (2006). The three causes are these: lack of data; lack of control of the input variables; uncertainty about the underlying probability distributions and correlations.

See also

• All models are wrong
• Cross-validation (statistics)
• Identifiability analysis
• Internal validity
• Model identification
• Model selection
• Overfitting
• Statistical conclusion validity
• Validity (statistics)
• Verification and validation

References

• Batzel, J. J.; Bachar, M.; Kappel, F., eds. (2013), Mathematical Modeling and Validation in Physiology, Springer.
• Ljung, L.; Guo, Lei (1997), "The role of model validation for assessing the size of the unmodeled dynamics", IEEE Transactions on Automatic Control, 42: 1230–1239, doi:10.1109/9.623084.
• National Research Council (2012), "Chapter 5: Model validation and prediction", Assessing the Reliability of Complex Models: Mathematical and statistical foundations of verification, validation, and uncertainty quantification, Washington, DC: National Academies Press, doi:10.17226/13395{{citation}}: CS1 maint: multiple names: authors list (link).

External links

• How can I tell if a model fits my data?  —Handbook of Statistical Methods (NIST)
• What are core statistical model validation techniques?  —Stack Exchange