Video-based face recognition using probabilistic appearance manifolds

Kuang Chih Lee, Jeffrey Ho, Ming Hsuan Yang, David Kriegman

Research output: Contribution to journalConference article

346 Citations (Scopus)

Abstract

This paper presents a novel method to model and recognize human faces in video sequences. Each registered person is represented by a low-dimensional appearance manifold in the ambient image space. The complex nonlinear appearance manifold expressed as a collection of subsets (named pose manifolds), and the connectivity among them. Each pose manifold is approximated by an affine plane. To construct this representation, exemplars are sampled from videos, and these exemplars are clustered with a K-means algorithm; each cluster is represented as a plane computed through principal component analysis (PCA). The connectivity between the pose manifolds encodes the transition probability between images in each of the pose manifold and is learned from a training video sequences. A maximum a posteriori formulation is presented for face recognition in test video sequences by integrating the likelihood that the input image comes from a particular pose manifold and the transition probability to this pose manifold from the previous frame. To recognize faces with partial occlusion, we introduce a weight mask into the process. Extensive experiments demonstrate that the proposed algorithm outperforms existing frame-based face recognition methods with temporal voting schemes.

Original languageEnglish
Pages (from-to)I/313-I/320
JournalProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
Volume1
Publication statusPublished - 2003 Sep 1
Event2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Madison, WI, United States
Duration: 2003 Jun 182003 Jun 20

Fingerprint

Face recognition
Principal component analysis
Masks
Experiments

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition

Cite this

@article{a57d0f6c8ba64931a288bd39447c0bd2,
title = "Video-based face recognition using probabilistic appearance manifolds",
abstract = "This paper presents a novel method to model and recognize human faces in video sequences. Each registered person is represented by a low-dimensional appearance manifold in the ambient image space. The complex nonlinear appearance manifold expressed as a collection of subsets (named pose manifolds), and the connectivity among them. Each pose manifold is approximated by an affine plane. To construct this representation, exemplars are sampled from videos, and these exemplars are clustered with a K-means algorithm; each cluster is represented as a plane computed through principal component analysis (PCA). The connectivity between the pose manifolds encodes the transition probability between images in each of the pose manifold and is learned from a training video sequences. A maximum a posteriori formulation is presented for face recognition in test video sequences by integrating the likelihood that the input image comes from a particular pose manifold and the transition probability to this pose manifold from the previous frame. To recognize faces with partial occlusion, we introduce a weight mask into the process. Extensive experiments demonstrate that the proposed algorithm outperforms existing frame-based face recognition methods with temporal voting schemes.",
author = "Lee, {Kuang Chih} and Jeffrey Ho and Yang, {Ming Hsuan} and David Kriegman",
year = "2003",
month = "9",
day = "1",
language = "English",
volume = "1",
pages = "I/313--I/320",
journal = "Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition",
issn = "1063-6919",
publisher = "IEEE Computer Society",

}

Video-based face recognition using probabilistic appearance manifolds. / Lee, Kuang Chih; Ho, Jeffrey; Yang, Ming Hsuan; Kriegman, David.

In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 01.09.2003, p. I/313-I/320.

Research output: Contribution to journalConference article

TY - JOUR

T1 - Video-based face recognition using probabilistic appearance manifolds

AU - Lee, Kuang Chih

AU - Ho, Jeffrey

AU - Yang, Ming Hsuan

AU - Kriegman, David

PY - 2003/9/1

Y1 - 2003/9/1

N2 - This paper presents a novel method to model and recognize human faces in video sequences. Each registered person is represented by a low-dimensional appearance manifold in the ambient image space. The complex nonlinear appearance manifold expressed as a collection of subsets (named pose manifolds), and the connectivity among them. Each pose manifold is approximated by an affine plane. To construct this representation, exemplars are sampled from videos, and these exemplars are clustered with a K-means algorithm; each cluster is represented as a plane computed through principal component analysis (PCA). The connectivity between the pose manifolds encodes the transition probability between images in each of the pose manifold and is learned from a training video sequences. A maximum a posteriori formulation is presented for face recognition in test video sequences by integrating the likelihood that the input image comes from a particular pose manifold and the transition probability to this pose manifold from the previous frame. To recognize faces with partial occlusion, we introduce a weight mask into the process. Extensive experiments demonstrate that the proposed algorithm outperforms existing frame-based face recognition methods with temporal voting schemes.

AB - This paper presents a novel method to model and recognize human faces in video sequences. Each registered person is represented by a low-dimensional appearance manifold in the ambient image space. The complex nonlinear appearance manifold expressed as a collection of subsets (named pose manifolds), and the connectivity among them. Each pose manifold is approximated by an affine plane. To construct this representation, exemplars are sampled from videos, and these exemplars are clustered with a K-means algorithm; each cluster is represented as a plane computed through principal component analysis (PCA). The connectivity between the pose manifolds encodes the transition probability between images in each of the pose manifold and is learned from a training video sequences. A maximum a posteriori formulation is presented for face recognition in test video sequences by integrating the likelihood that the input image comes from a particular pose manifold and the transition probability to this pose manifold from the previous frame. To recognize faces with partial occlusion, we introduce a weight mask into the process. Extensive experiments demonstrate that the proposed algorithm outperforms existing frame-based face recognition methods with temporal voting schemes.

UR - http://www.scopus.com/inward/record.url?scp=18244422049&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=18244422049&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:18244422049

VL - 1

SP - I/313-I/320

JO - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

JF - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

SN - 1063-6919

ER -