Faces are highly dynamic and deformable. Small motions convey our emotional state and intentions. Capturing nature 3D face shape in motion is the first step towards modeling facial motion. With such models we can animate realistic avatars and analyze videos of facial motion.
Our 4D face scanner employs 3 pairs of stereo cameras for computing shape and geometry with the assistance of custom speckle projectors and 3 color cameras and white-light panels for capturing texture, all of them arranged specifically to acquire the data needed to accurately reconstruct a 3D model of the human face in motion. The system can also be reconfigured to capture 3D hand shape over time.
We process the raw 3D scan data using our co-registration algorithm to bring all the scans into correspondence with a template mesh. With this we can model facial shape and motion over time to study the dynamics of the human face in ways that were not previously possible.
For example, we can obtain a 3D meshes of a person having a conversation and study how different emotions are conveyed by the motion of the face. These capabilities, together with the full body 3D models that we can acquire with our 4D Dynamic Scanner, enable us to observe how the expression and movement of our face are reflected in the motion and posture of the rest of our body and vice versa, providing new insights into the relationship between face and body movement.