Video-based face recognition using probabilistic appearance manifolds

Kuang Chih Lee, Jeffrey Ho, Ming Hsuan Yang, David Kriegman

Research output: Contribution to journalConference articlepeer-review

405 Citations (Scopus)


This paper presents a novel method to model and recognize human faces in video sequences. Each registered person is represented by a low-dimensional appearance manifold in the ambient image space. The complex nonlinear appearance manifold expressed as a collection of subsets (named pose manifolds), and the connectivity among them. Each pose manifold is approximated by an affine plane. To construct this representation, exemplars are sampled from videos, and these exemplars are clustered with a K-means algorithm; each cluster is represented as a plane computed through principal component analysis (PCA). The connectivity between the pose manifolds encodes the transition probability between images in each of the pose manifold and is learned from a training video sequences. A maximum a posteriori formulation is presented for face recognition in test video sequences by integrating the likelihood that the input image comes from a particular pose manifold and the transition probability to this pose manifold from the previous frame. To recognize faces with partial occlusion, we introduce a weight mask into the process. Extensive experiments demonstrate that the proposed algorithm outperforms existing frame-based face recognition methods with temporal voting schemes.

Original languageEnglish
Pages (from-to)I/313-I/320
JournalProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Publication statusPublished - 2003
Event2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Madison, WI, United States
Duration: 2003 Jun 182003 Jun 20

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition


Dive into the research topics of 'Video-based face recognition using probabilistic appearance manifolds'. Together they form a unique fingerprint.

Cite this