Joint Face Alignment with Non-Parametric Shape Models

Brandon M. Smith Li Zhang

University of Wisconsin, Madison



We present a joint face alignment technique that takes a set of images as input and produces a set of shape- and appearance-consistent face alignments as output. Our method is an extension of the recent localization method of Belhumeur et al., which combines the output of local detectors with a non-parametric set of face shape models. We are inspired by the recent joint alignment method of Zhao et al., which employs a modified Active Appearance Model (AAM) approach to align a batch of images. We introduce an approach for simultaneously optimizing both a local appearance constraint, which couples the local estimates between multiple images, and a global shape constraint, which couples landmarks and images across the image set. In video sequences, our method greatly improves the temporal stability of landmark estimates without compromising accuracy relative to ground truth.

Brandon M. Smith, Li Zhang. Joint Face Alignment with Non-Parametric Shape Models, European Conference on Computer Vision (ECCV), October, 2012. [Author's version: PDF 8.2 MB]
The original publication is available at, here.
This work is supported in part by NSF IIS-0845916, NSF IIS-0916441, a Sloan Research Fellowship, and a Packard Fellowship for Science and Engineering. Brandon Smith is also supported by an NSF graduate fellowship.
Supplemental Video
Download [MP4 28.2 MB]
Joint Alignment Video
Experimental Video Data - Added 2013-03-13
Update 2013-03-17: Now includes Matlab .mat data files, which provide the landmarks estimated using our algorithm for each video frame.
Download [ZIP 41.9 MB]