Depth Analogy: Data-Driven Approach for Single Image Depth Estimation Using Gradient Samples

Sunghwan Choi, Dongbo Min, Bumsub Ham, Youngjung Kim, Changjae Oh, Kwanghoon Sohn

Research output: Contribution to journalArticle

19 Citations (Scopus)

Abstract

Inferring scene depth from a single monocular image is a highly ill-posed problem in computer vision. This paper presents a new gradient-domain approach, called depth analogy, that makes use of analogy as a means for synthesizing a target depth field, when a collection of RGB-D image pairs is given as training data. Specifically, the proposed method employs a non-parametric learning process that creates an analogous depth field by sampling reliable depth gradients using visual correspondence established on training image pairs. Unlike existing data-driven approaches that directly select depth values from training data, our framework transfers depth gradients as reconstruction cues, which are then integrated by the Poisson reconstruction. The performance of most conventional approaches relies heavily on the training RGB-D data used in the process, and such a dependency severely degenerates the quality of reconstructed depth maps when the desired depth distribution of an input image is quite different from that of the training data, e.g., outdoor versus indoor scenes. Our key observation is that using depth gradients in the reconstruction is less sensitive to scene characteristics, providing better cues for depth recovery. Thus, our gradient-domain approach can support a great variety of training range datasets that involve substantial appearance and geometric variations. The experimental results demonstrate that our (depth) gradient-domain approach outperforms existing data-driven approaches directly working on depth domain, even when only uncorrelated training datasets are available.

Original languageEnglish
Article number7308054
Pages (from-to)5953-5966
Number of pages14
JournalIEEE Transactions on Image Processing
Volume24
Issue number12
DOIs
Publication statusPublished - 2015 Dec

Fingerprint

Computer vision
Sampling
Recovery

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Graphics and Computer-Aided Design

Cite this

Choi, Sunghwan ; Min, Dongbo ; Ham, Bumsub ; Kim, Youngjung ; Oh, Changjae ; Sohn, Kwanghoon. / Depth Analogy : Data-Driven Approach for Single Image Depth Estimation Using Gradient Samples. In: IEEE Transactions on Image Processing. 2015 ; Vol. 24, No. 12. pp. 5953-5966.
@article{fa614e1e25a443288ea41da023281aac,
title = "Depth Analogy: Data-Driven Approach for Single Image Depth Estimation Using Gradient Samples",
abstract = "Inferring scene depth from a single monocular image is a highly ill-posed problem in computer vision. This paper presents a new gradient-domain approach, called depth analogy, that makes use of analogy as a means for synthesizing a target depth field, when a collection of RGB-D image pairs is given as training data. Specifically, the proposed method employs a non-parametric learning process that creates an analogous depth field by sampling reliable depth gradients using visual correspondence established on training image pairs. Unlike existing data-driven approaches that directly select depth values from training data, our framework transfers depth gradients as reconstruction cues, which are then integrated by the Poisson reconstruction. The performance of most conventional approaches relies heavily on the training RGB-D data used in the process, and such a dependency severely degenerates the quality of reconstructed depth maps when the desired depth distribution of an input image is quite different from that of the training data, e.g., outdoor versus indoor scenes. Our key observation is that using depth gradients in the reconstruction is less sensitive to scene characteristics, providing better cues for depth recovery. Thus, our gradient-domain approach can support a great variety of training range datasets that involve substantial appearance and geometric variations. The experimental results demonstrate that our (depth) gradient-domain approach outperforms existing data-driven approaches directly working on depth domain, even when only uncorrelated training datasets are available.",
author = "Sunghwan Choi and Dongbo Min and Bumsub Ham and Youngjung Kim and Changjae Oh and Kwanghoon Sohn",
year = "2015",
month = "12",
doi = "10.1109/TIP.2015.2495261",
language = "English",
volume = "24",
pages = "5953--5966",
journal = "IEEE Transactions on Image Processing",
issn = "1057-7149",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "12",

}

Depth Analogy : Data-Driven Approach for Single Image Depth Estimation Using Gradient Samples. / Choi, Sunghwan; Min, Dongbo; Ham, Bumsub; Kim, Youngjung; Oh, Changjae; Sohn, Kwanghoon.

In: IEEE Transactions on Image Processing, Vol. 24, No. 12, 7308054, 12.2015, p. 5953-5966.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Depth Analogy

T2 - Data-Driven Approach for Single Image Depth Estimation Using Gradient Samples

AU - Choi, Sunghwan

AU - Min, Dongbo

AU - Ham, Bumsub

AU - Kim, Youngjung

AU - Oh, Changjae

AU - Sohn, Kwanghoon

PY - 2015/12

Y1 - 2015/12

N2 - Inferring scene depth from a single monocular image is a highly ill-posed problem in computer vision. This paper presents a new gradient-domain approach, called depth analogy, that makes use of analogy as a means for synthesizing a target depth field, when a collection of RGB-D image pairs is given as training data. Specifically, the proposed method employs a non-parametric learning process that creates an analogous depth field by sampling reliable depth gradients using visual correspondence established on training image pairs. Unlike existing data-driven approaches that directly select depth values from training data, our framework transfers depth gradients as reconstruction cues, which are then integrated by the Poisson reconstruction. The performance of most conventional approaches relies heavily on the training RGB-D data used in the process, and such a dependency severely degenerates the quality of reconstructed depth maps when the desired depth distribution of an input image is quite different from that of the training data, e.g., outdoor versus indoor scenes. Our key observation is that using depth gradients in the reconstruction is less sensitive to scene characteristics, providing better cues for depth recovery. Thus, our gradient-domain approach can support a great variety of training range datasets that involve substantial appearance and geometric variations. The experimental results demonstrate that our (depth) gradient-domain approach outperforms existing data-driven approaches directly working on depth domain, even when only uncorrelated training datasets are available.

AB - Inferring scene depth from a single monocular image is a highly ill-posed problem in computer vision. This paper presents a new gradient-domain approach, called depth analogy, that makes use of analogy as a means for synthesizing a target depth field, when a collection of RGB-D image pairs is given as training data. Specifically, the proposed method employs a non-parametric learning process that creates an analogous depth field by sampling reliable depth gradients using visual correspondence established on training image pairs. Unlike existing data-driven approaches that directly select depth values from training data, our framework transfers depth gradients as reconstruction cues, which are then integrated by the Poisson reconstruction. The performance of most conventional approaches relies heavily on the training RGB-D data used in the process, and such a dependency severely degenerates the quality of reconstructed depth maps when the desired depth distribution of an input image is quite different from that of the training data, e.g., outdoor versus indoor scenes. Our key observation is that using depth gradients in the reconstruction is less sensitive to scene characteristics, providing better cues for depth recovery. Thus, our gradient-domain approach can support a great variety of training range datasets that involve substantial appearance and geometric variations. The experimental results demonstrate that our (depth) gradient-domain approach outperforms existing data-driven approaches directly working on depth domain, even when only uncorrelated training datasets are available.

UR - http://www.scopus.com/inward/record.url?scp=84960193272&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84960193272&partnerID=8YFLogxK

U2 - 10.1109/TIP.2015.2495261

DO - 10.1109/TIP.2015.2495261

M3 - Article

AN - SCOPUS:84960193272

VL - 24

SP - 5953

EP - 5966

JO - IEEE Transactions on Image Processing

JF - IEEE Transactions on Image Processing

SN - 1057-7149

IS - 12

M1 - 7308054

ER -