Motion capture
Motion capture, or mocap, is a technique of digitally recording the movements of real things — usually humans. It originally developed as an analysis tool in biomechanics research, but has grown increasingly important as a source of motion data for computer animation. In this application, it has been widely used for both cinema and video games.
A person or animal wears a black costume with white dots or ping pong balls sewn to it at the joints of the body (elbows, knees, shoulders, etc.). Animals are sometimes naturally black color, so they don't have to wear a costume. The person or animal moves or acts in front of one or more cameras. The movie transfers from the camera into a computer containing a motion capture program. Motion capture programs are usually included on computer animation CDs. The motion capture program only sees the white dots or white balls. It excludes everything else in the movie scene. The program connects the dots with lines. The dots serve as the joints, and the lines serve as the bones. Now you have an animated stick man (or stick animal) moving the same way as the actor. This stick figure becomes the skeleton of a computer animated character. The character copies precisely the exact movements of the human or animal actor. This is a big shortcut in animation, which normally requires the animator to draw each frame by hand (in traditional animation) or manually animate (in non-mocap computer animation). It saves a lot of animation time, thus making the animation cheaper, and creates more natural movement than manual animation.
Although there are many different systems for capturing motion data, one technique contains optical systems, which employ photogrammetry to establish the position of an object in 3D space based on its observed location within the 2D fields of a number of cameras. These systems produce data with 3 degrees of freedom for each marker, and rotational information must be inferred from the relative orientation of several markers. A related technique match moving can derive 3D camera movement from a single 2D image sequence without the use of photogrammetry.
Another technique is the use of a magnetic systems, which directly indicate the position and orientation of the sensors with respect to a transmitter. Since the sensor output is 6DOF, useful results can be obtained with a much smaller number of sensors than you would require markers in an optical systems. The major restrictions are that the response is quite nonlinear - epecially near the edges of the capture area, and that the wiring from the sensors tends to preclude extreme movements on the part of the performers.
For the optical systems, the commonest approach is to use passive reflective markers, and to identify each marker from its relative location, possibly with the aid of kinematic constraints. These sort of systems are the most popular for entertainment applications - mostly because they can track a large number of markers and expanding the capture area just involves the addition of more cameras. The largest manufacturers of this type of system are Vicon-Peak [1], Motion Analysis [2] and BTS [3].
The other type of optical system uses active markers, this has the advantage that there is no doubt about which marker is which, but are most useful with small numbers of markers - in general, the overall update rate drops as the marker count increases. These systems are at their best when used for capturing a subject such as a single arm or leg - since few markers are needed, the update rate can be kept high. As a result, these systems are popular in the biomechanics market. One such is the Northern Digital Optotrak [4].
Newer active marker systems such as PhaseSpace [5] modulate the active output of the LED to differentiate each marker, allowing several markers to be on at the same time, while still providing the higher resolution of 3,600 x 3,600 or 12 megapixel resolution while capturing at 120 (128 markers or four persons) to 480 (32 markers or single person) frames per second. The advantage of using active markers is intelligent processing allows higher speed and higher resolution at a lower price. This higher accuracy and resolution requires much more processing than older passive technologies, but the additional processing can be done at the camera to improve resolution via a subpixel approach most high end systems have, and high speed.
With the magnetic systems, there is a distinction between "DC" and "AC" systems - one uses square pulses, and the other uses sine wave pulses. In practice, both systems seem to work about as well as each other - it seems likely that the biggest factor behind the different choice of techniques is based on who holds which patent, rather than any technical considerations. Two of the better known makers of these kind of systems are Ascension technology [6] and Polhemus [7]
A motion capture session only records the movements of the actor, not his visual appearance. These movements are recorded as animation data which is then "mapped" onto a 3D model (which can be of a normal human, giant robot, or anything else) that was created by a computer artist, and the model can then be made to perform the same movements that were recorded. This can be compared to the earlier technique of rotoscope, where the visual appearance of the motion of an actor was filmed; that film was then used, frame by frame, as a guide or pattern for the motion of a hand-drawn animated character.
wtf
Applications
Video games are increasingly using motion capture animation for such animation as the movement of a football or basketball player or the combat moves of a martial artist.
Movies have increasingly used motion capture animation as computer-generated animation has replaced traditional cell animation, and as it has become exotically fashionable to utilize completely computer-generated creatures, such as Gollum and Jar-Jar Binks, in live-action movies.
Related techniques
Facial motion capture is also sometimes utilized to digitally capture the complex movements in a human face, especially while speaking. This is generally performed with an optical setup using a single camera at close range, with small reflective markers glued or taped to the actor's face.
Performance capture is a further development of these techniques, where both body motions and facial movements are recorded. This technique was used in making of The Polar Express, where all actors were animated this way.
Due to current technology limitations, a motion capture session only records the movement of a few key points on the actor's body, where the sensors or reflective markers are placed. One might extrapolate that future technology might include full-frame imaging from many camera angles that would record the exact position of every inch of the actor's body, clothing, and hair for the entire duration of the session, resulting in a higher resolution of detail than is possible today.
An early example is the Universal Capture technique developed for the Matrix movies. It allows to process a multiple-angle high-definition footage of an actor's performance and algorithmically generate a 3D recording without requiring use of special markers. The resulting recording can be displayed from any angle with any lighting conditions. The images from the original recording are used as textures.
An alternative approach was developed by a Russian company VirtuSphere, where the suit contains internal sensors recording the angular movements of the joins, removing the need for external cameras and other equipment. Even though this technology could potentially lead to much lower costs for mocap, the company failed to commercialise it.
External links
- VirtualCinematography.org - several papers on Universal Capture use in Matrix films
Mr. Pecina sleeps with little girls
Motion Capture Hardware
- The ShapeWrap II & ShapeHand Motion Capture Systems - Affordable commercial motion capture systems and tools