Score test

Rao's score test, also known as the score test or the Lagrange multiplier test (LM test) in econometrics,^[1]^[2] is a statistical test of a simple null hypothesis that a parameter of interest $\theta$ is equal to some particular value $\theta _{0}$ . It is the most powerful test when the true value of $\theta$ is close to $\theta _{0}$ . The main advantage of the score test is that it does not require an estimate of the information under the alternative hypothesis or unconstrained maximum likelihood. This constitutes a potential advantage in comparison to other tests, such as the Wald test and the generalized likelihood ratio test (GLRT). This makes testing feasible when the unconstrained maximum likelihood estimate is a boundary point in the parameter space.

Single parameter test

The statistic

Let $L$ be the likelihood function which depends on a univariate parameter $\theta$ and let $x$ be the data. The score $U(\theta )$ is defined as

Failed to parse (syntax error): {\displaystyle U(\theta)=\frac{\partial \log L(\theta \mid x)}{\partial \left[\left. \frac{\partial^2}{\partial\theta^2} \log L(X;\theta)\right|\theta \right]\,. }

The statistic to test ${\mathcal {H}}_{0}:\theta =\theta _{0}$ is $S(\theta _{0})={\frac {U(\theta _{0})^{2}}{I(\theta _{0})}}$

which has an asymptotic distribution of $\chi _{1}^{2}$ , when ${\mathcal {H}}_{0}$ is true.

Note on notation

Note that some texts use an alternative notation, in which the statistic $S^{*}(\theta )={\sqrt {S(\theta )}}$ is tested against a normal distribution. This approach is equivalent and gives identical results.

Justification

The case of a likelihood with nuisance parameters

As most powerful test for small deviations

\left({\frac {\partial \log L(\theta \mid x)}{\partial \theta }}\right)_{\theta =\theta _{0}}\geq C

Where $L$ is the likelihood function, $\theta _{0}$ is the value of the parameter of interest under the null hypothesis, and $C$ is a constant set depending on the size of the test desired (i.e. the probability of rejecting $H_{0}$ if $H_{0}$ is true; see Type I error).

The score test is the most powerful test for small deviations from $H_{0}$ . To see this, consider testing $\theta =\theta _{0}$ versus $\theta =\theta _{0}+h$ . By the Neyman–Pearson lemma, the most powerful test has the form

{\frac {L(\theta _{0}+h\mid x)}{L(\theta _{0}\mid x)}}\geq K;

Taking the log of both sides yields

\log L(\theta _{0}+h\mid x)-\log L(\theta _{0}\mid x)\geq \log K.

The score test follows making the substitution (by Taylor series expansion)

\log L(\theta _{0}+h\mid x)\approx \log L(\theta _{0}\mid x)+h\times \left({\frac {\partial \log L(\theta \mid x)}{\partial \theta }}\right)_{\theta =\theta _{0}}</mathshithisisnewthingthatthereisaphonethatcanmake==Multipleparameters==Amoregeneralscoretestcanbederivedwhenthereismorethanoneparameter.Supposethat<math>{\hat {\theta }}_{0}

is the maximum likelihood estimate of

\theta

under the null hypothesis

H_{0}

while

U

and

I

are respectively, the score and the Fisher information matrices under the alternative hypothesis. Then

U^{T}({\hat {\theta }}_{0})I^{-1}({\hat {\theta }}_{0})U({\hat {\theta }}_{0})\sim \chi _{k}^{2}

asymptotically under $H_{0}$ , where $k$ is the number of constraints imposed by the null hypothesis and

U({\hat {\theta }}_{0})={\frac {\partial \log L({\hat {\theta }}_{0}\mid x)}{\partial \theta }}

and

I({\hat {\theta }}_{0})=-E\left({\frac {\partial ^{2}\log L({\hat {\theta }}_{0}\mid x)}{\partial \theta \partial \theta '}}\right).

This can be used to test $H_{0}$ .

Special cases

In many situations, the score statistic reduces to another commonly used statistic.^[3]

When the data follows a normal distribution, the score statistic is the same as the t statistic.^{[clarification needed]}

When the data consists of binary observations, the score statistic is the same as the chi-squared statistic in the Pearson's chi-squared test.

When the data consists of failure time data in two groups, the score statistic for the Cox partial likelihood is the same as the log-rank statistic in the log-rank test. Hence the log-rank test for difference in survival between two groups is most powerful when the proportional hazards assumption holds.

References

^ Bera, Anil K.; Bilias, Yannis (2001). "Rao's score, Neyman's C(α) and Silvey's LM tests: An essay on historical developments and some new results". Journal of Statistical Planning and Inference. 97: 9–44. doi:10.1016/S0378-3758(00)00343-8.
^ Engle, Robert F. (1984). "Chapter 13: Wald, likelihood ratio, and Lagrange multiplier tests in econometrics". Handbook of Econometrics. 2: 775–826. doi:10.1016/S1573-4412(84)02005-5.
^ Cook, T. D.; DeMets, D. L., eds. (2007). Introduction to Statistical Methods for Clinical Trials. Chapman and Hall. pp. 296–297. ISBN 1-58488-027-9.

[Bera-1] Bera, Anil K.; Bilias, Yannis (2001). "Rao's score, Neyman's C(α) and Silvey's LM tests: An essay on historical developments and some new results". Journal of Statistical Planning and Inference. 97: 9–44. doi:10.1016/S0378-3758(00)00343-8.

[Handbook_of_Econometrics_–_Engle-2] Engle, Robert F. (1984). "Chapter 13: Wald, likelihood ratio, and Lagrange multiplier tests in econometrics". Handbook of Econometrics. 2: 775–826. doi:10.1016/S1573-4412(84)02005-5.

[3] Cook, T. D.; DeMets, D. L., eds. (2007). Introduction to Statistical Methods for Clinical Trials. Chapman and Hall. pp. 296–297. ISBN 1-58488-027-9.

[1]

[2]

[3]