Single view-based 3D face reconstruction robust to self-occlusion

Youn Joo Lee, Sung Joo Lee, Kang Ryoung Park, Jaeik Jo, Jaihie Kim

Research output: Contribution to journalArticle

16 Citations (Scopus)

Abstract

State-of-the-art 3D morphable model (3DMM) is used widely for 3D face reconstruction based on a single image. However, this method has a high computational cost, and hence, a simplified 3D morphable model (S3DMM) was proposed as an alternative. Unlike the original 3DMM, S3DMM uses only a sparse 3D facial shape, and therefore, it incurs a lower computational cost. However, this method is vulnerable to self-occlusion due to head rotation. Therefore, we propose a solution to the self-occlusion problem in S3DMM-based 3D face reconstruction. This research is novel compared with previous works, in the following three respects. First, self-occlusion of the input face is detected automatically by estimating the head pose using a cylindrical head model. Second, a 3D model fitting scheme is designed based on selected visible facial feature points, which facilitates 3D face reconstruction without any effect from self-occlusion. Third, the reconstruction performance is enhanced by using the estimated pose as the initial pose parameter during the 3D model fitting process. The experimental results showed that the self-occlusion detection had high accuracy and our proposed method delivered a noticeable improvement in the 3D face reconstruction performance compared with previous methods.

Original languageEnglish
Article number176
JournalEurasip Journal on Advances in Signal Processing
Volume2012
Issue number1
DOIs
Publication statusPublished - 2012 Dec 1

Fingerprint

Costs

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Information Systems
  • Hardware and Architecture
  • Electrical and Electronic Engineering

Cite this

Lee, Youn Joo ; Lee, Sung Joo ; Park, Kang Ryoung ; Jo, Jaeik ; Kim, Jaihie. / Single view-based 3D face reconstruction robust to self-occlusion. In: Eurasip Journal on Advances in Signal Processing. 2012 ; Vol. 2012, No. 1.
@article{8182d7a821fb4daeba41da4292ad97f5,
title = "Single view-based 3D face reconstruction robust to self-occlusion",
abstract = "State-of-the-art 3D morphable model (3DMM) is used widely for 3D face reconstruction based on a single image. However, this method has a high computational cost, and hence, a simplified 3D morphable model (S3DMM) was proposed as an alternative. Unlike the original 3DMM, S3DMM uses only a sparse 3D facial shape, and therefore, it incurs a lower computational cost. However, this method is vulnerable to self-occlusion due to head rotation. Therefore, we propose a solution to the self-occlusion problem in S3DMM-based 3D face reconstruction. This research is novel compared with previous works, in the following three respects. First, self-occlusion of the input face is detected automatically by estimating the head pose using a cylindrical head model. Second, a 3D model fitting scheme is designed based on selected visible facial feature points, which facilitates 3D face reconstruction without any effect from self-occlusion. Third, the reconstruction performance is enhanced by using the estimated pose as the initial pose parameter during the 3D model fitting process. The experimental results showed that the self-occlusion detection had high accuracy and our proposed method delivered a noticeable improvement in the 3D face reconstruction performance compared with previous methods.",
author = "Lee, {Youn Joo} and Lee, {Sung Joo} and Park, {Kang Ryoung} and Jaeik Jo and Jaihie Kim",
year = "2012",
month = "12",
day = "1",
doi = "10.1186/1687-6180-2012-176",
language = "English",
volume = "2012",
journal = "Eurasip Journal on Advances in Signal Processing",
issn = "1687-6172",
publisher = "Springer Publishing Company",
number = "1",

}

Single view-based 3D face reconstruction robust to self-occlusion. / Lee, Youn Joo; Lee, Sung Joo; Park, Kang Ryoung; Jo, Jaeik; Kim, Jaihie.

In: Eurasip Journal on Advances in Signal Processing, Vol. 2012, No. 1, 176, 01.12.2012.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Single view-based 3D face reconstruction robust to self-occlusion

AU - Lee, Youn Joo

AU - Lee, Sung Joo

AU - Park, Kang Ryoung

AU - Jo, Jaeik

AU - Kim, Jaihie

PY - 2012/12/1

Y1 - 2012/12/1

N2 - State-of-the-art 3D morphable model (3DMM) is used widely for 3D face reconstruction based on a single image. However, this method has a high computational cost, and hence, a simplified 3D morphable model (S3DMM) was proposed as an alternative. Unlike the original 3DMM, S3DMM uses only a sparse 3D facial shape, and therefore, it incurs a lower computational cost. However, this method is vulnerable to self-occlusion due to head rotation. Therefore, we propose a solution to the self-occlusion problem in S3DMM-based 3D face reconstruction. This research is novel compared with previous works, in the following three respects. First, self-occlusion of the input face is detected automatically by estimating the head pose using a cylindrical head model. Second, a 3D model fitting scheme is designed based on selected visible facial feature points, which facilitates 3D face reconstruction without any effect from self-occlusion. Third, the reconstruction performance is enhanced by using the estimated pose as the initial pose parameter during the 3D model fitting process. The experimental results showed that the self-occlusion detection had high accuracy and our proposed method delivered a noticeable improvement in the 3D face reconstruction performance compared with previous methods.

AB - State-of-the-art 3D morphable model (3DMM) is used widely for 3D face reconstruction based on a single image. However, this method has a high computational cost, and hence, a simplified 3D morphable model (S3DMM) was proposed as an alternative. Unlike the original 3DMM, S3DMM uses only a sparse 3D facial shape, and therefore, it incurs a lower computational cost. However, this method is vulnerable to self-occlusion due to head rotation. Therefore, we propose a solution to the self-occlusion problem in S3DMM-based 3D face reconstruction. This research is novel compared with previous works, in the following three respects. First, self-occlusion of the input face is detected automatically by estimating the head pose using a cylindrical head model. Second, a 3D model fitting scheme is designed based on selected visible facial feature points, which facilitates 3D face reconstruction without any effect from self-occlusion. Third, the reconstruction performance is enhanced by using the estimated pose as the initial pose parameter during the 3D model fitting process. The experimental results showed that the self-occlusion detection had high accuracy and our proposed method delivered a noticeable improvement in the 3D face reconstruction performance compared with previous methods.

UR - http://www.scopus.com/inward/record.url?scp=84887047332&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84887047332&partnerID=8YFLogxK

U2 - 10.1186/1687-6180-2012-176

DO - 10.1186/1687-6180-2012-176

M3 - Article

AN - SCOPUS:84887047332

VL - 2012

JO - Eurasip Journal on Advances in Signal Processing

JF - Eurasip Journal on Advances in Signal Processing

SN - 1687-6172

IS - 1

M1 - 176

ER -