Proper lighting is a key element in developing a photorealistic computer-generated image. This paper introduces a novel approach for robustly extracting lighting conditions from an RGB-D (RGB + depth) image. Existing studies on lighting estimation have developed image analysis techniques by constraining the scope and condition of the target objects. For example, they have assumed that the objects have homogeneous surfaces, inter-reflections can be ignored, and their three-dimensional (3D) geometries consist of a noise-free mesh. These assumptions, however, are unrealistic; real objects often have complicated non-homogeneous surfaces, inter-reflections that affect a considerable portion of illumination, or unpredictable noise that can affect sensor measurements. To overcome these limitations, this study takes non-homogeneous surface objects into account in the inverse lighting framework via segment-based scene representation. Moreover, we employ outlier removal and appropriate region selection to achieve robust lighting estimation in the presence of inter-reflections and noise. We demonstrate the effectiveness of the proposed approach by conducting extensive experiments on synthetic and real RGB-D images.
Bibliographical noteFunding Information:
We thank three reviewers for their insightful comments and constructive suggestions, which greatly improve our paper. This work was supported by ICT R&D program of MSIP /IITP [ R7124-16-0004 , Development of Intelligent Interaction Technology Based on Context Awareness and Human Intention Understanding ], the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the MSIP ( NRF-2016R1A2B4016236 ), the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the University Information Technology Research Center support program (IITP-2017-2016-0-00288) supervised by the IITP (Institute for Information & communication Technology Promotion), and the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the IT Consilience Creative Program (IITP-2017-2017-0-01015) supervised by the IITP(Institute for Information & Communications Technology Promotion).
All Science Journal Classification (ASJC) codes
- Control and Systems Engineering
- Theoretical Computer Science
- Computer Science Applications
- Information Systems and Management
- Artificial Intelligence