Lichtfeld

The light field is a function that describes the amount of light travelling in every direction through every point in space. Michael Faraday was the first to propose (in an 1846 lecture entitled "Thoughts on Ray Vibrations") that light should be interpreted as a field, much like the magnetic fields on which he had been working for several years. The phrase light field was coined by Arun Gershun in a classic paper on the radiometric properties of light in three-dimensional space (1936). The phrase has been redefined by researchers in computer graphics to mean something slightly different. To understand this difference, we'll need a bit of terminology.

The 5D plenoptic function

Radiance L along a ray can be thought of as the amount of light traveling along all possible straight lines through a tube whose size is determined by its solid angle and cross-sectional area.

If we restrict ourselves to geometric optics, i.e. to incoherent light and to objects larger than the wavelength of light, then the fundamental carrier of light is a ray. The measure for the amount of light traveling along a ray is radiance, denoted by L and measured in watts (W) per steradian (sr) per meter squared (m²). The steradian is a measure of solid angle, and meters squared are used here as a measure of cross-sectional area, as shown at right.

Datei:Plenoptic-function-b.png

Parameterizing a ray in 3D space by position (x,y,z) and direction ( $\theta$ , $\phi$ ).

The radiance along all such rays in a region of three-dimensional space illuminated by an unchanging arrangement of lights is called the plenoptic function (Adelson 1991) (see plenoptic illumination function). Since rays in space can be parameterized by three coordinates, x, y, and z and two angles $\theta$ and $\phi$ , as shown at left, it is a five-dimensional function. (One can consider time, wavelength, and polarization angle as additional variables, yielding higher-dimensional functions.)

Datei:Gershun-irradiance-vectors.png

Summing the irradiance vectors

D_{1}

and

D_{2}

arising from two light sources

I_{1}

and

I_{2}

produces a resultant vector

D

having the magnitude and direction shown (from Gershun).

Like Adelson, Gershun defined the light field at each point in space as a 5D function. However, he treated it as an infinite collection of vectors, one per direction impinging on the point, with lengths proportional to their radiances. Equivalently, one can imagine an infinite collection of infinitesimal surfaces placed at that point, one per direction, with different values of irradiance assigned to each surface.

Integrating these vectors over any collection of lights, or over the entire sphere of directions, produces a single scalar value - the total irradiance at that point, and a resultant direction. The figure at right, reproduced from Gershun's paper, shows this calculation for the case of two light sources. In computer graphics, this vector-valued function of 3D space is called the vector irradiance field (Arvo, 1994). The vector direction at each point in the field can be interpreted as the orientation one would face a flat surface placed at that point to most brightly illuminate it.

The 4D light field

In a plenoptic function, if the region of interest contains a concave object (think of a cupped hand), then light leaving one point on the object may travel only a short distance before being blocked by another point on the object. No practical device could measure the function in such a region.

Radiance along a ray remains constant if there are no blockers.

However, if we restrict ourselves to locations outside the convex hull (think shrink-wrap) of the object, then we can measure the plenoptic function easily using a digital camera. Moreover, in this case the function contains redundant information, because the radiance along a ray remains constant from point to point along its length, as shown at left. In fact, the redundant information is exactly one dimension, leaving us with a four-dimensional function. Parry Moon dubbed this function the photic field (1981), while researchers in computer graphics call it the 4D light field (Levoy 1996; Gortler 1996). Formally, the 4D light field is defined as radiance along rays in empty space.

The set of rays in a light field can be parameterized in a variety of ways, a few of which are shown below. Of these, the most common is the two-plane parameterization shown at right (below). While this parameterization cannot represent all rays, for example rays parallel to the two planes if the planes are parallel to each other, it has the advantage of relating closely to the analytic geometry of perspective imaging. Indeed, a simple way to think about a two-plane light field is as a collection of perspective images of the st plane (and any objects that may lie astride or beyond it), each taken from an observer position on the uv plane. A light field parameterized this way is sometimes called a light slab.

Some alternative parameterizations of the 4D light field, which represents the flow of light through an empty region of three-dimensional space. *Left:* points on a plane or curved surface and directions leaving each point. *Center:* pairs of points on the surface of a sphere. *Right:* pairs of points on two planes in general (meaning any) position.

Devices for capturing light fields

Light fields are a fundamental representation for light. As such, there are as many ways of creating light fields as there are computer programs capable of creating images or instruments capable of capturing them.

In computer graphics, light fields are typically produced either by rendering a 3D model or by photographing a real scene. In either case, to produce a light field views must be obtained for a large collection of viewpoints. Depending on the parameterization employed, this collection will typically span some portion of a line, circle, plane, sphere, or other shape, although unstructured collections of viewpoints are also possible (Buehler 2001).

Devices for capturing light fields photographically include a moving handheld camera, an arc of cameras (as in the bullet time effect used in The Matrix, a dense array of cameras, or a handheld camera or microscope, or other optical system in which an array of microlenses has been inserted in the optical path.

Applications of light fields

Computational imaging refers to any image formation method that involves a digital computer. Many of these methods operate at visible wavelengths, and many of those produce light fields. As a result, listing all applications of light fields would require surveying all uses of computational imaging - in art, science, engineering, and medicine. In computer graphics, some selected applications are:

Light field rendering. By extracting appropriate 2D slices from the 4D light field of a scene, one can produce novel views of the scene (Levoy 1996; Gortler 1996). Depending on the parameterization of the light field and slices, these views might be perspective (see Perspective projection), orthographic (see Orthographic projection (geometry)), crossed-slit (Zomet 2003), multi-perspective (Rademacher 1998), or another type of projection.

Synthetic aperture photography. By integrating an appropriate 4D subset of the samples in a light field, one can approximate the view that would be captured by a camera having a finite (i.e. non-pinhole) aperture. Such a view has a finite depth of field. By shearing or warping the light field before performing this integration, one can focus on different fronto-parallel (Isaksen 2000) or oblique (Vaish 2005) planes in the scene.

Autostereoscopic display. By displaying a light field using technology that maps each sample to the appropriate ray in physical space, one obtains a stereo visual effect (without glasses, hence the word "autostereoscopic") akin to viewing the original scene. (See stereoscopy.) Example technologies include placing an array of lenslets over a high-resolution display screen or paper print in the manner of integral photography, projecting the imagery onto an array of lenslets using a video projector (Matusik 2004), or holography.

References

Theory

Adelson, E.H., Bergen, J.R. (1991). "The plenoptic function and the elements of early vision", In Computation Models of Visual Processing, M. Landy and J.A. Movshon, eds., MIT Press, Cambridge, 1991, pp. 3-20.

Arvo, J. (1994). "The Irradiance Jacobian for Partially Occluded Polyhedral Sources", Proc. ACM Siggraph, ACM Press, pp. 335-342.

Faraday, M., "Thoughts on Ray Vibrations", Philosophical Magazine, S.3, Vol XXVIII, N188, May 1846.

Gershun, A. (1936). "The Light Field", Moscow, 1936. Translated by P. Moon and G. Timoshenko in Journal of Mathematics and Physics, Vol. XVIII, MIT, 1939, pp. 51-151.

Gortler, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M. (1996). "The Lumigraph", Proc. ACM Siggraph, ACM Press, pp. 43-54.

Levoy, M., Hanrahan, P. (1996). "Light Field Rendering", Proc. ACM Siggraph, ACM Press, pp. 31-42.

Moon, P., Spencer, D.E. (1981). The Photic Field, MIT Press.

Applications

Buehler, C., Bosse, M., McMillan, L., Gortler, S., Cohen, M. (2001). "Unstructured Lumigraph rendering", Proc. ACM Siggraph, ACM Press.

Isaksen, A., McMillan, L., Gortler, S.J. (2000). "Dynamically Reparameterized Light Fields", Proc. ACM Siggraph, ACM Press, pp. 297-306.

Matusik, W., Pfister, H. (2004). ``3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes, Proc. ACM Siggraph, ACM Press.

Rademacher, P., Bishop, G. (1998). "Multiple-Center-of-Projection Images", Proc. ACM Siggraph, ACM Press.

Vaish, V., Garg, G., Talvala, E., Antunez, E., Wilburn, B., Horowitz, M., Levoy, M. (2005). "Synthetic Aperture Focusing using a Shear-Warp Factorization of the Viewing Transform", Proc. Workshop on Advanced 3D Imaging for Safety and Security, in conjunction with CVPR 2005.

Zomet, A., Feldman, D., Peleg, S., Weinshall, D. (2003). "Mosaicing new views: the crosssed-slits projection", IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 25, No. 6, June 2003, pp. 741-754.

External links