Additive smoothing

In the field of statistical language modeling and statistics additive smoothing is a technique used to smooth distribution $p(x)$ representing for example occurrences of a word $x$ in a text.

The additively smoothed distribution is defined as:

$p(x;\delta )={\frac {p(x)+\delta }{N+\delta |X|}}$

where $\delta$ is a number typically between 0 and 1, $X$ is the group of all sample groups (for example different words) and $N$ is the number of all samples $N=\sum {x\in X}p(x)$

Additive smoothing is sometimes referred as Lidstone smoothing.

References

• SF Chen, J Goodman (1996). An empirical study of smoothing techniques for language modeling. Proceedings of the 34th annual meeting on Association for Computational Linguistics table of contents