Log probability

In computer science, the use of log probabilities is a way to represent probabilities in a way which has several practical computational advantages over the standard use of approximated real numbers in the $[0,1]$ interval.

The log probability representation is to store a probability $x\in [0,1]$ as $x'=-\log(x)\in \mathbb {R}$. The product of probabilities $x\cdot y$ can be written as $x'+y'$; the sum of probabilities $x+y$ is more complex to express and is written as $-\log(e^{-x'}+e^{-y'})$. Note, however, that we can avoid to incur the cost of this computation by approximating the sum of two probabilities by the highest probability, at the expense of accuracy.

Representing probabilities in this way has two main advantages:

1. Speed. Since multiplication is more expensive than addition, taking the product of a high number of probabilities is faster if they are represented in log form. (The conversion to log form is expensive, but is only incurred once.)
2. Accuracy. The use of log probabilities improves numerical stability.

The use of log probabilities is widespread in several fields of computer science such as information theory and natural language processing.