Face Modeling

Overview

3d Modeling. Generating realistic 3D human face models and facial animations has been a persistent challenge in computer vision and graphics. We have developed a system that constructs textured 3D face models from videos with minimal user interaction. Our system takes a video sequence of a face with an ordinary video camera. After five manual clicks on two images to tell the system where the eye corners, nose tip and mouth corners are, the system automatically generates a realistic looking 3D human head model and the constructed model can immediately be animated (different poses, facial expressions and talking). Optionally, in order to achieve higher accuracy, a user can click an additional set of three points to indicate the lower part of the face. A user, with a PC and an ordinary camera, can use our system to generate his/her face model in a few minutes.

Facial Expression. We are also interested in facial expression mapping. Facial expressions exhibit not only facial feature motions, but also subtle changes in illumination and appearance (e.g., facial creases and wrinkles). These details are important visual cues, but they are difficult to synthesize. Traditional expression mapping techniques consider feature motions while the details in illumination changes are ignored. We propose to capture the illumination change of one person’s expression in what we call an expression ratio image (ERI). Together with geometric warping, we map an ERI to any other person’s face image to generate more expressive facial expressions.

Media Coverage

• Red Herring Magazine by Niall McKay (4/5/2001): Let’s play with Microsoft
• CBS Saturday Early Show with Renay San Miguel (4/21/2001)
• MicroNews: 3-D Talking Heads on the Horizon (1/12/2001)