Jump to content

Scoring algorithm

From Wikipedia, the free encyclopedia
This is an old revision of this page, as edited by 164.125.33.71 (talk) at 06:29, 9 October 2007 (Application to Linear Models). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

In statistics, Fisher's Scoring algorithm is a form of Newton's method used to solve maximum likelihood equations numerically.


Sketch of Derivation

Let be random variables, independent and identically distributed with twice differentiable p.d.f. , and we wish to calculate the maximum likelihood estimator (M.L.E.) of . First, suppose we have a starting point for our algorithm , and consider a Taylor expansion of the score function, , about :

,

where

is the observed information matrix at . Now, setting , using that and rearranging gives us:

.

We therefore use the algorithm

,

and under certain regularity conditions, it can be shown that .

Fisher Scoring

In practice, is usually replaced by , the Fisher information, thus giving us the Fisher Scoring Algorithm:

.


File:Example.jpg'''Bold text''''Bold text''''==Application to Linear Models==

The method of Fisher Scoring is often used in the theory of linear models. Suppose we have a standard linear model

,

where Failed to parse (unknown function "\si"): {\displaystyle \varepsilon_i \sim N(0,\si#REDIRECT [[#REDIRECT [[Insert text]]#REDIRECT [[#REDIRECT [[Insert text]]#REDIRECT [[<br />Insert text<br /><br /><br /><sub> <gallery> Subscript text </gallery><blockquote> {| class="wikitable" |- Block quote |} </blockquote></sub>]]]]]]gma^2)} independently; now suppose we want to estimate β. It can be shown[1] that

where is the M.L.E. of β. It is therefore desirable to find and use it as an estimator of β.

References

  1. ^ A.C. Davidson Statistical Models. Cambridge University Press (2003).