FTS (Facial Technology & Simulation)

A software & data collection targetting modeling and animation of 3D faces

Eftychios Sifakis - University of Wisconsin Madison

 

Overview

FTS - a Facial Technology & Simulation package will be an open-source, BSD licensed collection of data and algorithms, designed to facilitate the modeling and animation of photorealistic, anatomically faithful muscle-driven models of human faces. This package will include a host of techniques for finite element modeling and simulation, in addition to a subject-specific human face model, complete with muscles, high resolution surface geometry and 3D medical imagery.

Q & A

What is included in this release, other than simulation code?

All anatomical data, surface scans, skin textures, simulation meshes and animation data created for the purposes of the publications listed below will be included in the release. The simulation code itself is more generic in nature than what is strictly needed for simulating facial models, and would be well reusable for the simulation of other anatomical structures, or other soft elastic objects.

Any restrictions on the use of this data?

The geometric data included in this release can be used for any purpose, and in any context. The high-resolution skin model can freely be used for any task, including simulation, morphing, deformation, slicing, fracture, rendering etc. The code can be alterered and re-distributed in accordance with the BSD software license.

Will the code depend on a specific simulation engine?

The greater part of this project will leverage the PhysBAM simulation engine from Stanford University. Any components of PhysBAM that are essential for FTS to function will be also freely released under the same BSD license terms.

When will this public release take place?

Most components of the FTS project will become available through this webpage by Q1 2011

Video previews

Fully dynamic simulation, using embedded hybrid discretizations:

Embedded face simulation using hybrid geometries

Embedded mesh generation, by carving the flesh volume out of a background tetrahedral lattice:

An embedded face model is carved out of a block of material, then disected even more

Speech training, analysis and novel synthesis:

Estimation of muscle activations from sample spoken words Capture of facial expressions, which are translated into muscle activation values New words synthesized phoneme-by-phoneme from examples Synthesized words, blended with facial expressions A synthesized sentence, with external interactions (collisions) added to the simulation A longer completely synthetic speech segment

Embedded volumetric musculature:

Illustration of active embedded musculature

Facial animation and analysis from motion capture data:

Cross section of tetrahedral flesh simulation mesh Motion capture session used as input data Simulated (green) and motion captured (red) marker positions Facial expressions are generated by the action of muscles Comparison of the capture session, and the matching CG simulation A synthetic frowning expression A synthetic smiling expression An external collision object is inserted in the simulation using the pre-estimated activation values

 

Related publications

The simulation and modeling algorithms in the FTS public release will include implementations of algorithms from the following research papers:

Hybrid Simulation of Deformable Solids
E. Sifakis, T. Shinar, G. Irving and R. Fedkiw
ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA), edited by D. Metaxas and J. Popovic, 2007 [PDF]

Arbitrary Cutting of Deformable Tetrahedralized Objects
E. Sifakis, K. Der, and R. Fedkiw
ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA), edited by D. Metaxas and J. Popovic, 2007 [PDF]

Simulating Speech with a Physics-Based Facial Muscle Model
E. Sifakis, A. Selle, A. Robinson-Mosher and R. Fedkiw
ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA), edited by M.-P. Cani and J. O’Brien, 2006 [PDF]

Automatic Determination of Facial Muscle Activations from Sparse Motion Capture Marker Data
E. Sifakis, I. Neverov and R. Fedkiw
ACM Transactions on Graphics (SIGGRAPH Proceedings), TOG 24, pp. 417-425, 2005 [PDF]

Robust Quasistatic Finite Elements and Flesh Simulation
J. Teran, E. Sifakis, G. Irving and R. Fedkiw
ACM/Eurographics Symposium on Computer Animation (SCA),  edited by K. Anjyo and P. Faloutsos, pp. 181-190, 2005 [PDF]

Creating and simulating skeletal muscle from the Visible Human Data Set
J. Teran, E. Sifakis, S. Blemker, V. Ng Thow Hing, C. Lau and R. Fedkiw
IEEE Transactions on Visualization and Computer Graphics, 11, pp. 317-328, 2005 [PDF]

Fast 3D Muscle Simulations Using a New Quasistatic Invertible Finite-Element Algorithm
S. Blemker, J. Teran, E. Sifakis, R. Fedkiw and S. Delp
International Symposium on Computer Simulation in Biomechanics, 2005 [PDF]