Faces as light probes for relighting

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

A light probe is commonly used for measuring the illumination of a real scene. Instead of equipping a man-made light probe such as a mirror ball, we propose to use a face in images as a natural light probe. To that end, we construct a statistical reflectance model for faces and use this model to extract the lighting and the reflectance field of an input face. With an iterative procedure, we can obtain the lighting condition from an unknown face image. As a byproduct of this procedure, we also estimate the reflectance field of the same face. By identifying the lighting condition of scene, we can provide an effective solution for various practical applications. First, we can insert a virtual object seamlessly into a real scene by illuminating the virtual object under the lighting present in the real scene. Second, we can relight a face under an arbitrary lighting condition using the estimated reflectance fields. Third, we can swap two unknown faces using the estimates of both lighting and reflectance fields. Based on various experiments, we show that the proposed algorithm is an effective tool for many practical applications: inserting a virtual object into a real scene, face relighting, and face swapping.

Original languageEnglish
Article number077002
JournalOptical Engineering
Volume51
Issue number7
DOIs
Publication statusPublished - 2012 Jul 1

Fingerprint

light beams
illuminating
Lighting
reflectance
estimates
inserts
Byproducts
balls
Mirrors
illumination
mirrors
Experiments

All Science Journal Classification (ASJC) codes

  • Atomic and Molecular Physics, and Optics
  • Engineering(all)

Cite this

@article{b5eb9b8a0d7547c4923d6aa08aafa9f9,
title = "Faces as light probes for relighting",
abstract = "A light probe is commonly used for measuring the illumination of a real scene. Instead of equipping a man-made light probe such as a mirror ball, we propose to use a face in images as a natural light probe. To that end, we construct a statistical reflectance model for faces and use this model to extract the lighting and the reflectance field of an input face. With an iterative procedure, we can obtain the lighting condition from an unknown face image. As a byproduct of this procedure, we also estimate the reflectance field of the same face. By identifying the lighting condition of scene, we can provide an effective solution for various practical applications. First, we can insert a virtual object seamlessly into a real scene by illuminating the virtual object under the lighting present in the real scene. Second, we can relight a face under an arbitrary lighting condition using the estimated reflectance fields. Third, we can swap two unknown faces using the estimates of both lighting and reflectance fields. Based on various experiments, we show that the proposed algorithm is an effective tool for many practical applications: inserting a virtual object into a real scene, face relighting, and face swapping.",
author = "Hyunjung Shim",
year = "2012",
month = "7",
day = "1",
doi = "10.1117/1.OE.51.7.077002",
language = "English",
volume = "51",
journal = "Optical Engineering",
issn = "0091-3286",
publisher = "SPIE",
number = "7",

}

Faces as light probes for relighting. / Shim, Hyunjung.

In: Optical Engineering, Vol. 51, No. 7, 077002, 01.07.2012.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Faces as light probes for relighting

AU - Shim, Hyunjung

PY - 2012/7/1

Y1 - 2012/7/1

N2 - A light probe is commonly used for measuring the illumination of a real scene. Instead of equipping a man-made light probe such as a mirror ball, we propose to use a face in images as a natural light probe. To that end, we construct a statistical reflectance model for faces and use this model to extract the lighting and the reflectance field of an input face. With an iterative procedure, we can obtain the lighting condition from an unknown face image. As a byproduct of this procedure, we also estimate the reflectance field of the same face. By identifying the lighting condition of scene, we can provide an effective solution for various practical applications. First, we can insert a virtual object seamlessly into a real scene by illuminating the virtual object under the lighting present in the real scene. Second, we can relight a face under an arbitrary lighting condition using the estimated reflectance fields. Third, we can swap two unknown faces using the estimates of both lighting and reflectance fields. Based on various experiments, we show that the proposed algorithm is an effective tool for many practical applications: inserting a virtual object into a real scene, face relighting, and face swapping.

AB - A light probe is commonly used for measuring the illumination of a real scene. Instead of equipping a man-made light probe such as a mirror ball, we propose to use a face in images as a natural light probe. To that end, we construct a statistical reflectance model for faces and use this model to extract the lighting and the reflectance field of an input face. With an iterative procedure, we can obtain the lighting condition from an unknown face image. As a byproduct of this procedure, we also estimate the reflectance field of the same face. By identifying the lighting condition of scene, we can provide an effective solution for various practical applications. First, we can insert a virtual object seamlessly into a real scene by illuminating the virtual object under the lighting present in the real scene. Second, we can relight a face under an arbitrary lighting condition using the estimated reflectance fields. Third, we can swap two unknown faces using the estimates of both lighting and reflectance fields. Based on various experiments, we show that the proposed algorithm is an effective tool for many practical applications: inserting a virtual object into a real scene, face relighting, and face swapping.

UR - http://www.scopus.com/inward/record.url?scp=84885344708&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84885344708&partnerID=8YFLogxK

U2 - 10.1117/1.OE.51.7.077002

DO - 10.1117/1.OE.51.7.077002

M3 - Article

AN - SCOPUS:84885344708

VL - 51

JO - Optical Engineering

JF - Optical Engineering

SN - 0091-3286

IS - 7

M1 - 077002

ER -