Score test

In statistics , the Lagrange multiplier test (LM test) is one of three classical principles of hypothesis testing, together with the Wald test and the likelihood-ratio test, for testing a null hypothesis $H_{0}:\mathbf {c} (\theta )=\mathbf {0}$ for a parameter of interest $\theta$. The basic idea of the LM test is that if the restricted estimator is near the maximum of the likelihood function, the gradient of the likelihood function—known as the score—evaluated at the restricted estimator should be close to zero, and in fact asymptotically follows a normal distribution with mean zero. This result has first been proved by C. R. Rao in 1948, leading to the alternative name score test. An alternative and numerically identical version of the test was derived by S. D. Silvey in 1959, using the vector of Lagrange multipliers associated with the constraint in the Lagrangian expression of the constrained likelihood function, which led to the test's more commonly used name.

The main advantage of the LM test over the Wald test and likelihood ratio test is that the LM test only requires the computation of the restricted estimator. This makes testing feasible when the unconstrained maximum likelihood estimate is a boundary point in the parameter space.

Single parameter test

The statistic

Let $L$ be the likelihood function which depends on a univariate parameter $\theta$ and let $x$ be the data. The score $U(\theta )$ is defined as

$U(\theta )={\frac {\partial \log L(\theta \mid x)}{\partial \theta }}.$

The Fisher information is

$I(\theta )=-\operatorname {E} \left[\left.{\frac {\partial ^{2}}{\partial \theta ^{2}}}\log L(X;\theta )\right|\theta \right]\,.$

The statistic to test ${\mathcal {H}}_{0}:\theta =\theta _{0}$ is $S(\theta _{0})={\frac {U(\theta _{0})^{2}}{I(\theta _{0})}}$

which has an asymptotic distribution of $\chi _{1}^{2}$, when ${\mathcal {H}}_{0}$ is true.

Note on notation

Note that some texts use an alternative notation, in which the statistic $S^{*}(\theta )={\sqrt {S(\theta )}}$ is tested against a normal distribution. This approach is equivalent and gives identical results.

As most powerful test for small deviations

$\left({\frac {\partial \log L(\theta \mid x)}{\partial \theta }}\right)_{\theta =\theta _{0}}\geq C$

Where $L$ is the likelihood function, $\theta _{0}$ is the value of the parameter of interest under the null hypothesis, and $C$ is a constant set depending on the size of the test desired (i.e. the probability of rejecting $H_{0}$ if $H_{0}$ is true; see Type I error).

The score test is the most powerful test for small deviations from $H_{0}$. To see this, consider testing $\theta =\theta _{0}$ versus $\theta =\theta _{0}+h$. By the Neyman–Pearson lemma, the most powerful test has the form

${\frac {L(\theta _{0}+h\mid x)}{L(\theta _{0}\mid x)}}\geq K;$

Taking the log of both sides yields

$\log L(\theta _{0}+h\mid x)-\log L(\theta _{0}\mid x)\geq \log K.$

The score test follows making the substitution (by Taylor series expansion)

$\log L(\theta _{0}+h\mid x)\approx \log L(\theta _{0}\mid x)+h\times \left({\frac {\partial \log L(\theta \mid x)}{\partial \theta }}\right)_{\theta =\theta _{0}}$

and identifying the $C$ above with $\log(K)$.

Relationship with other hypothesis tests

The likelihood ratio test, the Wald test, and the Score test are asymptotically equivalent tests of hypotheses. When testing nested models, the statistics for each test converge to a Chi-squared distribution with degrees of freedom equal to the difference in degrees of freedom in the two models.

Multiple parameters

A more general score test can be derived when there is more than one parameter. Suppose that ${\hat {\theta }}_{0}$ is the maximum likelihood estimate of $\theta$ under the null hypothesis $H_{0}$ while $U$ and $I$ are respectively, the score and the Fisher information matrices under the alternative hypothesis. Then

$U^{T}({\hat {\theta }}_{0})I^{-1}({\hat {\theta }}_{0})U({\hat {\theta }}_{0})\sim \chi _{k}^{2}$

asymptotically under $H_{0}$, where $k$ is the number of constraints imposed by the null hypothesis and

$U({\hat {\theta }}_{0})={\frac {\partial \log L({\hat {\theta }}_{0}\mid x)}{\partial \theta }}$

and

$I({\hat {\theta }}_{0})=-E\left({\frac {\partial ^{2}\log L({\hat {\theta }}_{0}\mid x)}{\partial \theta \partial \theta '}}\right).$

This can be used to test $H_{0}$.

Special cases

In many situations, the score statistic reduces to another commonly used statistic.

When the data follows a normal distribution, the score statistic is the same as the t statistic.

When the data consists of binary observations, the score statistic is the same as the chi-squared statistic in the Pearson's chi-squared test.

When the data consists of failure time data in two groups, the score statistic for the Cox partial likelihood is the same as the log-rank statistic in the log-rank test. Hence the log-rank test for difference in survival between two groups is most powerful when the proportional hazards assumption holds.

See also

• Fisher information
• Uniformly most powerful test
• Score (statistics)
• Sup-LM test

Further reading

• Buse, A. (1982). "The Likelihood Ratio, Wald, and Lagrange Multiplier Tests: An Expository Note". The American Statistician. 36 (3a): 153–157. doi:10.1080/00031305.1982.10482817.
• Gallant, A. Ronald (1997). An Introduction to Econometric Theory. Princeton: Princeton University Press. pp. 179–181. ISBN 0-691-01645-3.