Statistical signal processing

Statistical signal processing is an approach to signal processing which treats signals as stochastic processes, utilizing their statistical properties to perform signal processing tasks. Statistical techniques are widely used in signal processing applications. For example, one can model the probability distribution of noise incurred when photographing an image, and construct techniques based on this model to reduce the noise in the resulting image.

Examples of statistical signal models

A classic problem in signal processing is estimating a signal, or its underlying parameters, from noisy observations. This is typically accomplished using either as a Bayesian or a frequentist model. We provide a classic example for each of these models.

Noise reduction

Main article: Wiener filter

Let $x(t)$ be an unknown, random, zero-mean, normally-distributed, wide-sense stationary process. We would like to estimate $x(t)$ from noisy measurements $y(t)=x(t)+w(t)$, where $w(t)$ is a noise process, uncorrelated with $x(t)$, which is likewise zero-mean, normally-distributed, and wide-sense stationary. Then, the optimal estimator is obtained by passing $y(t)$ through a linear filter whose frequency response is given by

$H(\omega )={\frac {S_{xx}(\omega )}{S_{xx}(\omega )+S_{ww}(\omega )}}$

where $S_{xx}(\omega )$ and $S_{ww}(\omega )$ are the spectral densities of $x(t)$ and $w(t)$, respectively.

Parameter estimation

Main article: Autoregressive model § Calculation of the AR parameters

Many types of signals can be modeled using a small number of unknown parameters. This can be useful, for example, in signal compression (only the parameters need be stored, rather than the entire signal), as well as signal analysis (the parameters may hint at underlying characteristics of the model generating the signals).

As an example, suppose a discrete-time signal $x[n]$ is modeled as an autoregressive process,

$x[n]=a_{1}x[n-1]+a_{2}x[n-2]+\cdots +a_{k}x[n-k]+w[n]$

where $w[n]$ is additive white Gaussian noise and $a_{1},\ldots ,a_{k}$ are unknown deterministic parameters (the autoregression parameters). Then, from observations of $x[n]$, it is of interest to estimate the autoregression parameters. This may be done using the method of least squares, or using the more computationally efficient Yule-Walker equations.

Further reading

• Scharf, Louis L. (1991). Statistical signal processing: detection, estimation, and time series analysis. Boston: Addison–Wesley. ISBN 0-201-19038-9. OCLC 61160161.
• P Stoica, R Moses (2005). Spectral Analysis of Signals (PDF). NJ: Prentice Hall.
• Kay, Steven M. (1993). Fundamentals of Statistical Signal Processing. Upper Saddle River, New Jersey: Prentice Hall. ISBN 0-13-345711-7. OCLC 26504848.
• Papoulis, Athanasios (1991). Probability, Random Variables, and Stochastic Processes (third ed.). McGraw-Hill. ISBN 0-07-100870-5.
• Kainam Thomas Wong [1]: Statistical Signal Processing lecture notes at the University of Waterloo, Canada.
• Ali H. Sayed, Adaptive Filters, Wiley, NJ, 2008, ISBN 978-0-470-25388-5.
• Thomas Kailath, Ali H. Sayed, and Babak Hassibi, Linear Estimation, Prentice-Hall, NJ, 2000, ISBN 978-0-13-022464-4.