Image processor

The image processing engine, or short image processor, is - next to the optics and the image sensor - one of the most important components of a digital camera and plays a vital role in creating the digital image.

The range of tasks the image processing engine needs to perform is pretty complex.

The photodiodes employed in an image sensor are colour-blind by nature: they can only record shades of grey. To get colour into the picture, they are covered with different colour filters: red, green and blue (RGB) - the Bayer filter (also called mosaic or array) - named after its inventor. As each photodiode records the colour information for exactly one pixel of the image, without an image processor there would be a green pixel next to each red and blue pixel. (Actually, with most sensors there are two green for each blue and red diodes.)

The image processing engine comprises a combination of hardware (processors) and software (algorithms). The image processor gathers the luminance and chrominance information from the individual pixels and uses it to compute/interpolate the correct colour and brightness values for each pixel. If it does this well, the result is an image with natural and pleasing colours, balanced contrast and sharpness .

This process, however, is quite a complex one and involves a number of different operations. Its success depends largely on the "intelligence" of the algorithms applied.

Getting the colours right
As stated above, the image processor evaluates the colour and brightness data of a given pixel, compares them with the data of its neighbouring pixels and processes these using a complex algorithm to get the correct colour and brightness value for this pixel. But the image processor also assesses the whole picture to guess at the correct distribution of contrast. By adjusting the gamma value (heightening or lowering the contrast range of an image's mid-tones) subtle tonal gradations, such as in human skin or the blue of the sky, come out much more realistic.

Getting rid of noise
Noise is a phenomenon found in any electronic circuitry. In digital photography its effect is often visible as random spots of obviously wrong colour in an otherwise smoothly-coloured area. Noise increases with temperature and exposure times. When higher ISO settings are chosen the electronic signal in the image sensor is amplified, which at the same time increases the noise level, leading to a lower signal-to-noise ratio. The image processor attempts to separate the noise from the image information and to remove it. This can be quite a challenge, as the image may contain areas with fine textures that when treated as noise may lose some of its definition.

Getting smooth and sharp edges
As the colour and brightness values for each pixel are interpolated some image softening is applied to even out fuzziness that may occur in the process. In order not to lose the impression of depth, clarity and fine details, sharpening of contours and edges is needed. The image processor therefore needs to be able to detect edges correctly and to reproduce them smoothly and without over-sharpening.

Getting the job done quickly
With the ever more increasing pixel count in image sensors, the image processor's speed becomes equally more important: Photographers don't want to wait for the camera's image processor to complete its job before they can carry on shooting - they don't even want to notice some processing is going on inside the camera. Therefore, they need to be optimised to cope with the higher data volume in the same or even a shorter period of time.

See also