Distribution-free maximum likelihood for binary responses

Distribution Free maximum Likelihood for Binary Responses

In this article, let’s take the latent utility model [1] as an example for the binary response model. The intuition of the latent utility model is that respondents will pick up the choice which will give the highest utility for her. Because the utility is not observable, it is assumed that the latent utility is linear with the some explanatory variables which affects the utility of the choice to the respondent and there is an additive response error capturing the randomness of the choice-making process. In this model, the choice is: $Y_{t}=1[\mathrm {X} _{1}t\beta +\varepsilon _{1}>\mathrm {X} _{2}t\beta +\varepsilon _{2}]$, where $\mathrm {X} _{1}t,\mathrm {X} _{2}t$ are two vectors of the explanatory covariates, $\varepsilon _{1}and\varepsilon _{2}$ are i.i.d response errors,

$\mathrm {X} _{1},t,\beta +\varepsilon _{1}$ and $\mathrm {X} _{2},t\beta +\varepsilon _{2}$ are latent utility of choosing choice 1 and 2. Then the log likelihood function can be given as:

$Q=\sum _{i-1}^{N}Y_{t}\centerdot \log(P[\mathrm {X} _{1},t\beta -\mathrm {X} _{2}t\beta >\varepsilon _{2}-\varepsilon _{1}])+(1-Y_{t})\centerdot \log(1-P[X_{1}t\beta -X_{2}t\beta >\varepsilon _{2}-\varepsilon _{1}])$

If some distributional assumption about the response error is imposed, then the log likelihood function [2] will have specific close form representation. For instance, if the response error is assumed to be distributed as: $N(0,\sigma ^{2})$ , then the likelihood function can be rewritten as:

$Q=\sum _{i-1}^{N}Y_{t}\centerdot \log {\biggl (}\Phi [{\frac {X_{1,t}\beta -X_{2,t}\beta }{\surd 2\sigma }}]{\biggr )}+(1-Y_{t})\centerdot \log {\Bigl (}\Phi [{\frac {X_{2,t}\beta -X_{1,t}\beta }{\surd 2\sigma }}]{\biggr )}$

where $\Phi (.)$ is the CDF for standard normal distribution. Here, even if $\Phi (.)$ doesn't have a closed form of representation, its derivative does. Therefore, the Maximum Likelihood Estimation can be explicitly computed by solving the first order condition. Alternatively, if the response error is assumed to be distributed as Gumbel $(0,\sigma ^{2})$, then the log-likelihood function can be rewritten as:

$Q=\sum _{i-1}^{N}Y_{t}\cdot \log {\biggl (}F[{\frac {X_{1,t}\beta -X_{2,t}\beta }{\sigma }}]{\biggr )}+(1-Y_{t})\centerdot \log {\biggl (}F[{\frac {X_{2,t}\beta -X_{1,t}\beta }{\sigma }}]{\Biggr )}$

where $F(.)$ is the CDF for the standard logistic distribution, which has a closed form representation.

Both of the models above are based on the distribution assumption about the response error term. Adding specific distribution assumption into the model can make the model computationally tractable due to the existence of the closed form representation. But if the distribution of the error term is mis-sepcified, the estimates based on the distribution assumption will be inconsistent. To get more robust estimator, models which don’t depend on the distribution assumption can be used. The basic idea of the distribution-free model is to replace the two probability term in the log-likelihood function with other weights. The general form of the log-likelihood function can written as:

$Q=\sum _{i-1}^{N}Y_{t}\cdot \log(W_{1}(X_{1}t\beta ,\mathrm {X} _{2},t\beta ))+(1-Y_{t})\log(W_{0}(X_{1},t,\beta ,X_{2},t\beta ))$

For instance, Manski (1975) [3] proposed a discrete weighting scheme for multi-response model, in the binary context which can be represented as:

where: $W_{1}(X_{1},t\beta ,\mathrm {X} _{2}\beta )=w_{1}[X,t\beta -X_{2}t\beta >0]+w_{0}1[X_{1}t\beta -X_{2}t\beta <0]$,

$W_{0}(X_{1}t\beta ,X_{2}t\beta )=1-W_{1}(X_{1}t\beta ,X_{2},t\beta )$

and $w_{1}$ and $w_{0}$ are two constants in (0,1). The intuition of this weighting scheme is that the probability of the choice depends on the relative order of the certainty part of the utility. Under the discrete weighting scheme, the estimator, which is also called Maximum Score Estimator, doesn’t have very nice asymptotic property [4], and Horowitz (1992) [5] proposed a smoothed weighting scheme, which can be represented as:

$W_{1}(X_{1},t\beta ,X_{2},t\beta )=K[X_{1}t\beta -X_{2},t\beta ],W_{0}(X_{1},t\beta ,X_{2},t\beta )=1-K[X_{1},t\beta -X_{2}t\beta ]$

where the weight function K (.) has to satisfy the following conditions:

(1) $|K(\cdot )|$ is bounded over R;

(2)$limK(u)=0$ and $limK(u)=1;$

$\upsilon \rightarrow \infty$ $\upsilon \rightarrow \infty$

(3) ${\dot {K}}(u)={\dot {K}}(-u)$

Here, the weight function is analogous to a CDF but can be more general and flexible than the weight functions in the models based on specific distribution assumption. The estimator under this weighting scheme is also called Smoothed Maximum Score Estimator. Usually, it is more computationally tractable than the Maximum Score Estimator for its smoothness and it is also more robust than the estimator based on the distribution assumptions.

References

1. Joan Walker, Moshe Ben-Akiva, Generalized random utility model, Mathematical Social Sciences, Volume 43, Issue 3, July 2002, Pages 303-343, ISSN 0165-4896.
2. Wooldridge, J. (2002): Econometric Analysis of Cross Section and Panel Data, MIT Press, Cambridge, Mass, pp 457-460.
3. Charles F. Manski (1975), “Maximum Score Estimation of the Stochastic Utility Model of Choice”, Journal of Econometrics 3, pp. 205-228.
4.  Jeankyung Kim; David Pollard (1990), “Cube Root Asymptotics”, The Annals of Statistics 1, pp.191-219.
5. Joel L. Horowitz (1992), “A Smoothed Maximum Score Estimator for the Binary Response Model”, Econometrica 3, pp. 505-531.

External links

• www.example.com