An example-based face relighting

Hyunjung Shim, Tsuhan Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we propose a new face relighting algorithm powered by a large database of face images captured under various known lighting conditions (a Multi-PIE database). Key insight of our algorithm is that a face can be represented by an assemble of patches from many other faces. The algorithm finds the most similar face patches in the database in terms of the lighting and the appearance. By assembling the matched patches, we can visualize the input face under various lighting conditions. Unlike existing face relighting algorithms, we neither use any kinds of face model nor make a physical assumption. Instead, our algorithm is a data-driven approach, synthesizing the appearance of the image patch using the appearance of the example patch. Using a data-driven approach, we can account for various intrinsic facial features including the non-Lambertian skin properties as well as the hair. Also, our algorithm is insensitive to the face misalignment. We demonstrate the performance of our algorithm by face relighting and face recognition experiments. Especially, the synthesized results show that the proposed algorithm can successfully handle various intrinsic features of an input face. Also, from the face recognition experiment, we show that our method is comparable to the most recent face relighting work.

Original languageEnglish
Title of host publicationProceedings of SPIE-IS and T Electronic Imaging - The Engineering Reality of Virtual Reality 2012
PublisherSPIE
ISBN (Print)9780819489364
DOIs
Publication statusPublished - 2012 Jan 1
EventThe Engineering Reality of Virtual Reality 2012 - Burlingame, CA, United States
Duration: 2012 Jan 242012 Jan 25

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume8289
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X

Other

OtherThe Engineering Reality of Virtual Reality 2012
CountryUnited States
CityBurlingame, CA
Period12/1/2412/1/25

Fingerprint

Face
Patch
illuminating
Lighting
Face recognition
Face Recognition
Data-driven
hair
assembling
misalignment
Skin
Experiments
Misalignment
Experiment
Demonstrate

All Science Journal Classification (ASJC) codes

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Cite this

Shim, H., & Chen, T. (2012). An example-based face relighting. In Proceedings of SPIE-IS and T Electronic Imaging - The Engineering Reality of Virtual Reality 2012 [82890B] (Proceedings of SPIE - The International Society for Optical Engineering; Vol. 8289). SPIE. https://doi.org/10.1117/12.908480
Shim, Hyunjung ; Chen, Tsuhan. / An example-based face relighting. Proceedings of SPIE-IS and T Electronic Imaging - The Engineering Reality of Virtual Reality 2012. SPIE, 2012. (Proceedings of SPIE - The International Society for Optical Engineering).
@inproceedings{7e6aebf9914040bca0ae90907e8c8181,
title = "An example-based face relighting",
abstract = "In this paper, we propose a new face relighting algorithm powered by a large database of face images captured under various known lighting conditions (a Multi-PIE database). Key insight of our algorithm is that a face can be represented by an assemble of patches from many other faces. The algorithm finds the most similar face patches in the database in terms of the lighting and the appearance. By assembling the matched patches, we can visualize the input face under various lighting conditions. Unlike existing face relighting algorithms, we neither use any kinds of face model nor make a physical assumption. Instead, our algorithm is a data-driven approach, synthesizing the appearance of the image patch using the appearance of the example patch. Using a data-driven approach, we can account for various intrinsic facial features including the non-Lambertian skin properties as well as the hair. Also, our algorithm is insensitive to the face misalignment. We demonstrate the performance of our algorithm by face relighting and face recognition experiments. Especially, the synthesized results show that the proposed algorithm can successfully handle various intrinsic features of an input face. Also, from the face recognition experiment, we show that our method is comparable to the most recent face relighting work.",
author = "Hyunjung Shim and Tsuhan Chen",
year = "2012",
month = "1",
day = "1",
doi = "10.1117/12.908480",
language = "English",
isbn = "9780819489364",
series = "Proceedings of SPIE - The International Society for Optical Engineering",
publisher = "SPIE",
booktitle = "Proceedings of SPIE-IS and T Electronic Imaging - The Engineering Reality of Virtual Reality 2012",
address = "United States",

}

Shim, H & Chen, T 2012, An example-based face relighting. in Proceedings of SPIE-IS and T Electronic Imaging - The Engineering Reality of Virtual Reality 2012., 82890B, Proceedings of SPIE - The International Society for Optical Engineering, vol. 8289, SPIE, The Engineering Reality of Virtual Reality 2012, Burlingame, CA, United States, 12/1/24. https://doi.org/10.1117/12.908480

An example-based face relighting. / Shim, Hyunjung; Chen, Tsuhan.

Proceedings of SPIE-IS and T Electronic Imaging - The Engineering Reality of Virtual Reality 2012. SPIE, 2012. 82890B (Proceedings of SPIE - The International Society for Optical Engineering; Vol. 8289).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - An example-based face relighting

AU - Shim, Hyunjung

AU - Chen, Tsuhan

PY - 2012/1/1

Y1 - 2012/1/1

N2 - In this paper, we propose a new face relighting algorithm powered by a large database of face images captured under various known lighting conditions (a Multi-PIE database). Key insight of our algorithm is that a face can be represented by an assemble of patches from many other faces. The algorithm finds the most similar face patches in the database in terms of the lighting and the appearance. By assembling the matched patches, we can visualize the input face under various lighting conditions. Unlike existing face relighting algorithms, we neither use any kinds of face model nor make a physical assumption. Instead, our algorithm is a data-driven approach, synthesizing the appearance of the image patch using the appearance of the example patch. Using a data-driven approach, we can account for various intrinsic facial features including the non-Lambertian skin properties as well as the hair. Also, our algorithm is insensitive to the face misalignment. We demonstrate the performance of our algorithm by face relighting and face recognition experiments. Especially, the synthesized results show that the proposed algorithm can successfully handle various intrinsic features of an input face. Also, from the face recognition experiment, we show that our method is comparable to the most recent face relighting work.

AB - In this paper, we propose a new face relighting algorithm powered by a large database of face images captured under various known lighting conditions (a Multi-PIE database). Key insight of our algorithm is that a face can be represented by an assemble of patches from many other faces. The algorithm finds the most similar face patches in the database in terms of the lighting and the appearance. By assembling the matched patches, we can visualize the input face under various lighting conditions. Unlike existing face relighting algorithms, we neither use any kinds of face model nor make a physical assumption. Instead, our algorithm is a data-driven approach, synthesizing the appearance of the image patch using the appearance of the example patch. Using a data-driven approach, we can account for various intrinsic facial features including the non-Lambertian skin properties as well as the hair. Also, our algorithm is insensitive to the face misalignment. We demonstrate the performance of our algorithm by face relighting and face recognition experiments. Especially, the synthesized results show that the proposed algorithm can successfully handle various intrinsic features of an input face. Also, from the face recognition experiment, we show that our method is comparable to the most recent face relighting work.

UR - http://www.scopus.com/inward/record.url?scp=84899085091&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84899085091&partnerID=8YFLogxK

U2 - 10.1117/12.908480

DO - 10.1117/12.908480

M3 - Conference contribution

AN - SCOPUS:84899085091

SN - 9780819489364

T3 - Proceedings of SPIE - The International Society for Optical Engineering

BT - Proceedings of SPIE-IS and T Electronic Imaging - The Engineering Reality of Virtual Reality 2012

PB - SPIE

ER -

Shim H, Chen T. An example-based face relighting. In Proceedings of SPIE-IS and T Electronic Imaging - The Engineering Reality of Virtual Reality 2012. SPIE. 2012. 82890B. (Proceedings of SPIE - The International Society for Optical Engineering). https://doi.org/10.1117/12.908480