A statistical framework for image-based relighting

Hyunjung Shim, Tsuhan Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

With image-based relighting (IBL), one can render realistic relit images of a scene without prior knowledge of object geometry in the scene. However, traditional IBL methods require a large number of basis images, each corresponding to a lighting pattern, to estimate the surface reflectance function (SRF) of the scene. In this paper, we present a statistical approach to estimating the SRF which requires fewer basis images. We formulate the SRF estimation problem in a signal reconstruction framework. We use the principal component analysis (PCA, [1]) to show that the most effective lighting patterns for the data acquisition process are the eigenvectors of the covariance matrix of the SRFs, corresponding to the largest eigenvalues. In addition, we show that for typical SRFs, especially when the objects have Lambertian surfaces, DCT-based lighting patterns perform as well as the optimal PCA-based lighting patterns. We compare SRF estimation performance of the statistical approach with traditional IBL techniques. Experimental results show that the statistical approach can achieve better performance with fewer basis images.

Original languageEnglish
Title of host publication2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '05 - Proceedings - Image and Multidimensional Signal Processing Multimedia Signal Processing
VolumeII
DOIs
Publication statusPublished - 2005 Dec 1
Event2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '05 - Philadelphia, PA, United States
Duration: 2005 Mar 182005 Mar 23

Other

Other2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '05
CountryUnited States
CityPhiladelphia, PA
Period05/3/1805/3/23

Fingerprint

Lighting
Signal reconstruction
Covariance matrix
Eigenvalues and eigenfunctions
Principal component analysis
Data acquisition
Geometry

All Science Journal Classification (ASJC) codes

  • Software
  • Signal Processing
  • Electrical and Electronic Engineering

Cite this

Shim, H., & Chen, T. (2005). A statistical framework for image-based relighting. In 2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '05 - Proceedings - Image and Multidimensional Signal Processing Multimedia Signal Processing (Vol. II). [1415599] https://doi.org/10.1109/ICASSP.2005.1415599
Shim, Hyunjung ; Chen, Tsuhan. / A statistical framework for image-based relighting. 2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '05 - Proceedings - Image and Multidimensional Signal Processing Multimedia Signal Processing. Vol. II 2005.
@inproceedings{0547bf63aa5744eba48ccd58878fd69e,
title = "A statistical framework for image-based relighting",
abstract = "With image-based relighting (IBL), one can render realistic relit images of a scene without prior knowledge of object geometry in the scene. However, traditional IBL methods require a large number of basis images, each corresponding to a lighting pattern, to estimate the surface reflectance function (SRF) of the scene. In this paper, we present a statistical approach to estimating the SRF which requires fewer basis images. We formulate the SRF estimation problem in a signal reconstruction framework. We use the principal component analysis (PCA, [1]) to show that the most effective lighting patterns for the data acquisition process are the eigenvectors of the covariance matrix of the SRFs, corresponding to the largest eigenvalues. In addition, we show that for typical SRFs, especially when the objects have Lambertian surfaces, DCT-based lighting patterns perform as well as the optimal PCA-based lighting patterns. We compare SRF estimation performance of the statistical approach with traditional IBL techniques. Experimental results show that the statistical approach can achieve better performance with fewer basis images.",
author = "Hyunjung Shim and Tsuhan Chen",
year = "2005",
month = "12",
day = "1",
doi = "10.1109/ICASSP.2005.1415599",
language = "English",
isbn = "0780388747",
volume = "II",
booktitle = "2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '05 - Proceedings - Image and Multidimensional Signal Processing Multimedia Signal Processing",

}

Shim, H & Chen, T 2005, A statistical framework for image-based relighting. in 2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '05 - Proceedings - Image and Multidimensional Signal Processing Multimedia Signal Processing. vol. II, 1415599, 2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '05, Philadelphia, PA, United States, 05/3/18. https://doi.org/10.1109/ICASSP.2005.1415599

A statistical framework for image-based relighting. / Shim, Hyunjung; Chen, Tsuhan.

2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '05 - Proceedings - Image and Multidimensional Signal Processing Multimedia Signal Processing. Vol. II 2005. 1415599.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - A statistical framework for image-based relighting

AU - Shim, Hyunjung

AU - Chen, Tsuhan

PY - 2005/12/1

Y1 - 2005/12/1

N2 - With image-based relighting (IBL), one can render realistic relit images of a scene without prior knowledge of object geometry in the scene. However, traditional IBL methods require a large number of basis images, each corresponding to a lighting pattern, to estimate the surface reflectance function (SRF) of the scene. In this paper, we present a statistical approach to estimating the SRF which requires fewer basis images. We formulate the SRF estimation problem in a signal reconstruction framework. We use the principal component analysis (PCA, [1]) to show that the most effective lighting patterns for the data acquisition process are the eigenvectors of the covariance matrix of the SRFs, corresponding to the largest eigenvalues. In addition, we show that for typical SRFs, especially when the objects have Lambertian surfaces, DCT-based lighting patterns perform as well as the optimal PCA-based lighting patterns. We compare SRF estimation performance of the statistical approach with traditional IBL techniques. Experimental results show that the statistical approach can achieve better performance with fewer basis images.

AB - With image-based relighting (IBL), one can render realistic relit images of a scene without prior knowledge of object geometry in the scene. However, traditional IBL methods require a large number of basis images, each corresponding to a lighting pattern, to estimate the surface reflectance function (SRF) of the scene. In this paper, we present a statistical approach to estimating the SRF which requires fewer basis images. We formulate the SRF estimation problem in a signal reconstruction framework. We use the principal component analysis (PCA, [1]) to show that the most effective lighting patterns for the data acquisition process are the eigenvectors of the covariance matrix of the SRFs, corresponding to the largest eigenvalues. In addition, we show that for typical SRFs, especially when the objects have Lambertian surfaces, DCT-based lighting patterns perform as well as the optimal PCA-based lighting patterns. We compare SRF estimation performance of the statistical approach with traditional IBL techniques. Experimental results show that the statistical approach can achieve better performance with fewer basis images.

UR - http://www.scopus.com/inward/record.url?scp=33646773238&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33646773238&partnerID=8YFLogxK

U2 - 10.1109/ICASSP.2005.1415599

DO - 10.1109/ICASSP.2005.1415599

M3 - Conference contribution

AN - SCOPUS:33646773238

SN - 0780388747

SN - 9780780388741

VL - II

BT - 2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '05 - Proceedings - Image and Multidimensional Signal Processing Multimedia Signal Processing

ER -

Shim H, Chen T. A statistical framework for image-based relighting. In 2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '05 - Proceedings - Image and Multidimensional Signal Processing Multimedia Signal Processing. Vol. II. 2005. 1415599 https://doi.org/10.1109/ICASSP.2005.1415599