Multimodal Face-Pose Estimation With Multitask Manifold Deep Learning

Chaoqun Hong, Jun Yu, Jian Zhang, Xiongnan Jin, Kyong Ho Lee

Research output: Contribution to journalArticle

56 Citations (Scopus)


Face-pose estimation aims at estimating the gazing direction with two-dimensional face images. It gives important communicative information and visual saliency. However, it is challenging because of lights, background, face orientations, and appearance visibility. Therefore, a descriptive representation of face images and mapping it to poses are critical. In this paper, we use multimodal data and propose a novel face-pose estimation framework named multitask manifold deep learning (M2DL). It is based on feature extraction with improved convolutional neural networks (CNNs) and multimodal mapping relationship with multitask learning. In the proposed CNNs, manifold regularized convolutional layers learn the relationship between outputs of neurons in a low-rank space. Besides, in the proposed mapping relationship learning method, different modals of face representations are naturally combined by applying multitask learning with incoherent sparse and low-rank learning with a least-squares loss. Experimental results on three challenging benchmark datasets demonstrate the performance of M2DL.

Original languageEnglish
Article number8554134
Pages (from-to)3952-3961
Number of pages10
JournalIEEE Transactions on Industrial Informatics
Issue number7
Publication statusPublished - 2019 Jul

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Information Systems
  • Computer Science Applications
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Multimodal Face-Pose Estimation With Multitask Manifold Deep Learning'. Together they form a unique fingerprint.

  • Cite this