Robust approach to inverse lighting using RGB-D images

Junsuk Choe, Hyunjung Shim

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Proper lighting is a key element in developing a photorealistic computer-generated image. This paper introduces a novel approach for robustly extracting lighting conditions from an RGB-D (RGB + depth) image. Existing studies on lighting estimation have developed image analysis techniques by constraining the scope and condition of the target objects. For example, they have assumed that the objects have homogeneous surfaces, inter-reflections can be ignored, and their three-dimensional (3D) geometries consist of a noise-free mesh. These assumptions, however, are unrealistic; real objects often have complicated non-homogeneous surfaces, inter-reflections that affect a considerable portion of illumination, or unpredictable noise that can affect sensor measurements. To overcome these limitations, this study takes non-homogeneous surface objects into account in the inverse lighting framework via segment-based scene representation. Moreover, we employ outlier removal and appropriate region selection to achieve robust lighting estimation in the presence of inter-reflections and noise. We demonstrate the effectiveness of the proposed approach by conducting extensive experiments on synthetic and real RGB-D images.

Original languageEnglish
Pages (from-to)73-94
Number of pages22
JournalInformation sciences
Volume438
DOIs
Publication statusPublished - 2018 Apr 1

Fingerprint

Lighting
Meshfree
Image Analysis
Outlier
Illumination
Sensor
Three-dimensional
Image analysis
Target
Object
Demonstrate
Experiment
Geometry
Sensors
Experiments

All Science Journal Classification (ASJC) codes

  • Software
  • Control and Systems Engineering
  • Theoretical Computer Science
  • Computer Science Applications
  • Information Systems and Management
  • Artificial Intelligence

Cite this

@article{463eb10b956a4d909d8c50efa8e83a3d,
title = "Robust approach to inverse lighting using RGB-D images",
abstract = "Proper lighting is a key element in developing a photorealistic computer-generated image. This paper introduces a novel approach for robustly extracting lighting conditions from an RGB-D (RGB + depth) image. Existing studies on lighting estimation have developed image analysis techniques by constraining the scope and condition of the target objects. For example, they have assumed that the objects have homogeneous surfaces, inter-reflections can be ignored, and their three-dimensional (3D) geometries consist of a noise-free mesh. These assumptions, however, are unrealistic; real objects often have complicated non-homogeneous surfaces, inter-reflections that affect a considerable portion of illumination, or unpredictable noise that can affect sensor measurements. To overcome these limitations, this study takes non-homogeneous surface objects into account in the inverse lighting framework via segment-based scene representation. Moreover, we employ outlier removal and appropriate region selection to achieve robust lighting estimation in the presence of inter-reflections and noise. We demonstrate the effectiveness of the proposed approach by conducting extensive experiments on synthetic and real RGB-D images.",
author = "Junsuk Choe and Hyunjung Shim",
year = "2018",
month = "4",
day = "1",
doi = "10.1016/j.ins.2018.01.049",
language = "English",
volume = "438",
pages = "73--94",
journal = "Information Sciences",
issn = "0020-0255",
publisher = "Elsevier Inc.",

}

Robust approach to inverse lighting using RGB-D images. / Choe, Junsuk; Shim, Hyunjung.

In: Information sciences, Vol. 438, 01.04.2018, p. 73-94.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Robust approach to inverse lighting using RGB-D images

AU - Choe, Junsuk

AU - Shim, Hyunjung

PY - 2018/4/1

Y1 - 2018/4/1

N2 - Proper lighting is a key element in developing a photorealistic computer-generated image. This paper introduces a novel approach for robustly extracting lighting conditions from an RGB-D (RGB + depth) image. Existing studies on lighting estimation have developed image analysis techniques by constraining the scope and condition of the target objects. For example, they have assumed that the objects have homogeneous surfaces, inter-reflections can be ignored, and their three-dimensional (3D) geometries consist of a noise-free mesh. These assumptions, however, are unrealistic; real objects often have complicated non-homogeneous surfaces, inter-reflections that affect a considerable portion of illumination, or unpredictable noise that can affect sensor measurements. To overcome these limitations, this study takes non-homogeneous surface objects into account in the inverse lighting framework via segment-based scene representation. Moreover, we employ outlier removal and appropriate region selection to achieve robust lighting estimation in the presence of inter-reflections and noise. We demonstrate the effectiveness of the proposed approach by conducting extensive experiments on synthetic and real RGB-D images.

AB - Proper lighting is a key element in developing a photorealistic computer-generated image. This paper introduces a novel approach for robustly extracting lighting conditions from an RGB-D (RGB + depth) image. Existing studies on lighting estimation have developed image analysis techniques by constraining the scope and condition of the target objects. For example, they have assumed that the objects have homogeneous surfaces, inter-reflections can be ignored, and their three-dimensional (3D) geometries consist of a noise-free mesh. These assumptions, however, are unrealistic; real objects often have complicated non-homogeneous surfaces, inter-reflections that affect a considerable portion of illumination, or unpredictable noise that can affect sensor measurements. To overcome these limitations, this study takes non-homogeneous surface objects into account in the inverse lighting framework via segment-based scene representation. Moreover, we employ outlier removal and appropriate region selection to achieve robust lighting estimation in the presence of inter-reflections and noise. We demonstrate the effectiveness of the proposed approach by conducting extensive experiments on synthetic and real RGB-D images.

UR - http://www.scopus.com/inward/record.url?scp=85041459784&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85041459784&partnerID=8YFLogxK

U2 - 10.1016/j.ins.2018.01.049

DO - 10.1016/j.ins.2018.01.049

M3 - Article

VL - 438

SP - 73

EP - 94

JO - Information Sciences

JF - Information Sciences

SN - 0020-0255

ER -