Natural facial and head behavior recognition using dictionary of motion primitives

Qun Shi; Norimichi Ukita; Ming Hsuan Yang

doi:10.1587/transinf.2017EDP7128

Natural facial and head behavior recognition using dictionary of motion primitives

Qun Shi, Norimichi Ukita, Ming Hsuan Yang

College of Computing

Research output: Contribution to journal › Article › peer-review

Abstract

This paper proposes a natural facial and head behavior recognition method using hybrid dynamical systems. Most existing facial and head behavior recognition methods focus on analyzing deliberately displayed prototypical emotion patterns rather than complex and spontaneous facial and head behaviors in natural conversation environments. We first capture spatio-temporal features on important facial parts via dense feature extraction. Next, we cluster the spatio-temporal features using hybrid dynamical systems, and construct a dictionary of motion primitives to cover all possible elemental motion dynamics accounting for facial and head behaviors. With this dictionary, the facial and head behavior can be interpreted into a distribution of motion primitives. This interpretation is robust against different rhythms of dynamic patterns in complex and spontaneous facial and head behaviors. We evaluate the proposed approach under natural tele-communication scenarios, and achieve promising results. Furthermore, the proposed method also performs favorably against the state-of-the-art methods on three benchmark databases.

Original language	English
Pages (from-to)	2993-3000
Number of pages	8
Journal	IEICE Transactions on Information and Systems
Volume	E100D
Issue number	12
DOIs	https://doi.org/10.1587/transinf.2017EDP7128
Publication status	Published - 2017 Dec

Bibliographical note

Funding Information:
This paper presented facial and head behavior recognition using dynamical systems, which consist of three steps, i.e., dynamic feature extraction, construction of dictionary based on motion primitives, and classification. We propose a novel dictionary based on motion primitives and show its effectiveness for recognition using statistical analysis without temporal alignment. Experimental results on four datasets demonstrate the effectiveness of the proposed method in recognizing natural and exerted facial and head behaviors. Our future work includes automatic extraction of video segments in each of which a natural facial and head behavior is observed in a long video sequence (e.g., video spotting). This work was supported by Yanmar Innovation Lab. 2112.

Publisher Copyright:
Copyright © 2017 The Institute of Electronics, Information and Communication Engineers.

All Science Journal Classification (ASJC) codes

Software
Hardware and Architecture
Computer Vision and Pattern Recognition
Electrical and Electronic Engineering
Artificial Intelligence

Access to Document

10.1587/transinf.2017EDP7128

Cite this

@article{178650bfe48c4b7e99a04f5437194d56,

title = "Natural facial and head behavior recognition using dictionary of motion primitives",

abstract = "This paper proposes a natural facial and head behavior recognition method using hybrid dynamical systems. Most existing facial and head behavior recognition methods focus on analyzing deliberately displayed prototypical emotion patterns rather than complex and spontaneous facial and head behaviors in natural conversation environments. We first capture spatio-temporal features on important facial parts via dense feature extraction. Next, we cluster the spatio-temporal features using hybrid dynamical systems, and construct a dictionary of motion primitives to cover all possible elemental motion dynamics accounting for facial and head behaviors. With this dictionary, the facial and head behavior can be interpreted into a distribution of motion primitives. This interpretation is robust against different rhythms of dynamic patterns in complex and spontaneous facial and head behaviors. We evaluate the proposed approach under natural tele-communication scenarios, and achieve promising results. Furthermore, the proposed method also performs favorably against the state-of-the-art methods on three benchmark databases.",

author = "Qun Shi and Norimichi Ukita and Yang, {Ming Hsuan}",

note = "Funding Information: This paper presented facial and head behavior recognition using dynamical systems, which consist of three steps, i.e., dynamic feature extraction, construction of dictionary based on motion primitives, and classification. We propose a novel dictionary based on motion primitives and show its effectiveness for recognition using statistical analysis without temporal alignment. Experimental results on four datasets demonstrate the effectiveness of the proposed method in recognizing natural and exerted facial and head behaviors. Our future work includes automatic extraction of video segments in each of which a natural facial and head behavior is observed in a long video sequence (e.g., video spotting). This work was supported by Yanmar Innovation Lab. 2112. Publisher Copyright: Copyright {\textcopyright} 2017 The Institute of Electronics, Information and Communication Engineers.",

year = "2017",

month = dec,

doi = "10.1587/transinf.2017EDP7128",

language = "English",

volume = "E100D",

pages = "2993--3000",

journal = "IEICE Transactions on Information and Systems",

issn = "0916-8532",

publisher = "Maruzen Co., Ltd/Maruzen Kabushikikaisha",

number = "12",

}

TY - JOUR

T1 - Natural facial and head behavior recognition using dictionary of motion primitives

AU - Shi, Qun

AU - Ukita, Norimichi

AU - Yang, Ming Hsuan

N1 - Funding Information: This paper presented facial and head behavior recognition using dynamical systems, which consist of three steps, i.e., dynamic feature extraction, construction of dictionary based on motion primitives, and classification. We propose a novel dictionary based on motion primitives and show its effectiveness for recognition using statistical analysis without temporal alignment. Experimental results on four datasets demonstrate the effectiveness of the proposed method in recognizing natural and exerted facial and head behaviors. Our future work includes automatic extraction of video segments in each of which a natural facial and head behavior is observed in a long video sequence (e.g., video spotting). This work was supported by Yanmar Innovation Lab. 2112. Publisher Copyright: Copyright © 2017 The Institute of Electronics, Information and Communication Engineers.

PY - 2017/12

Y1 - 2017/12

N2 - This paper proposes a natural facial and head behavior recognition method using hybrid dynamical systems. Most existing facial and head behavior recognition methods focus on analyzing deliberately displayed prototypical emotion patterns rather than complex and spontaneous facial and head behaviors in natural conversation environments. We first capture spatio-temporal features on important facial parts via dense feature extraction. Next, we cluster the spatio-temporal features using hybrid dynamical systems, and construct a dictionary of motion primitives to cover all possible elemental motion dynamics accounting for facial and head behaviors. With this dictionary, the facial and head behavior can be interpreted into a distribution of motion primitives. This interpretation is robust against different rhythms of dynamic patterns in complex and spontaneous facial and head behaviors. We evaluate the proposed approach under natural tele-communication scenarios, and achieve promising results. Furthermore, the proposed method also performs favorably against the state-of-the-art methods on three benchmark databases.

AB - This paper proposes a natural facial and head behavior recognition method using hybrid dynamical systems. Most existing facial and head behavior recognition methods focus on analyzing deliberately displayed prototypical emotion patterns rather than complex and spontaneous facial and head behaviors in natural conversation environments. We first capture spatio-temporal features on important facial parts via dense feature extraction. Next, we cluster the spatio-temporal features using hybrid dynamical systems, and construct a dictionary of motion primitives to cover all possible elemental motion dynamics accounting for facial and head behaviors. With this dictionary, the facial and head behavior can be interpreted into a distribution of motion primitives. This interpretation is robust against different rhythms of dynamic patterns in complex and spontaneous facial and head behaviors. We evaluate the proposed approach under natural tele-communication scenarios, and achieve promising results. Furthermore, the proposed method also performs favorably against the state-of-the-art methods on three benchmark databases.

UR - http://www.scopus.com/inward/record.url?scp=85038405479&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85038405479&partnerID=8YFLogxK

U2 - 10.1587/transinf.2017EDP7128

DO - 10.1587/transinf.2017EDP7128

M3 - Article

AN - SCOPUS:85038405479

SN - 0916-8532

VL - E100D

SP - 2993

EP - 3000

JO - IEICE Transactions on Information and Systems

JF - IEICE Transactions on Information and Systems

IS - 12

ER -

Natural facial and head behavior recognition using dictionary of motion primitives

Abstract

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this