Mean square quantization error
Appearance
Mean square quantization error is a figure of merit for the process of analog to digital conversion.
As the input is varied, the input's value is recorded when the digital output changes. For each digital output, the input's difference from ideal is normalized to the value of the least significant bit, then squared, summed, and normalized to the number of samples.