Structure Selective Depth Superresolution for RGB-D Cameras

Youngjung Kim, Bumsub Ham, Changjae Oh, Kwanghoon Sohn

Research output: Contribution to journalArticle

11 Citations (Scopus)

Abstract

This paper describes a method for high-quality depth superresolution. The standard formulations of image-guided depth upsampling, using simple joint filtering or quadratic optimization, lead to texture copying and depth bleeding artifacts. These artifacts are caused by inherent discrepancy of structures in data from different sensors. Although there exists some correlation between depth and intensity discontinuities, they are different in distribution and formation. To tackle this problem, we formulate an optimization model using a nonconvex regularizer. A nonlocal affinity established in a high-dimensional feature space is used to offer precisely localized depth boundaries. We show that the proposed method iteratively handles differences in structure between depth and intensity images. This property enables reducing texture copying and depth bleeding artifacts significantly on a variety of range data sets. We also propose a fast alternating direction method of multipliers algorithm to solve our optimization problem. Our solver shows a noticeable speed up compared with the conventional majorize-minimize algorithm. Extensive experiments with synthetic and real-world data sets demonstrate that the proposed method is superior to the existing methods.

Original languageEnglish
Article number7547322
Pages (from-to)5227-5238
Number of pages12
JournalIEEE Transactions on Image Processing
Volume25
Issue number11
DOIs
Publication statusPublished - 2016 Nov 1

Fingerprint

Copying
Cameras
Textures
Sensors
Experiments

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Graphics and Computer-Aided Design

Cite this

@article{3cc3ca6f2f1241679de9f8ab839253f8,
title = "Structure Selective Depth Superresolution for RGB-D Cameras",
abstract = "This paper describes a method for high-quality depth superresolution. The standard formulations of image-guided depth upsampling, using simple joint filtering or quadratic optimization, lead to texture copying and depth bleeding artifacts. These artifacts are caused by inherent discrepancy of structures in data from different sensors. Although there exists some correlation between depth and intensity discontinuities, they are different in distribution and formation. To tackle this problem, we formulate an optimization model using a nonconvex regularizer. A nonlocal affinity established in a high-dimensional feature space is used to offer precisely localized depth boundaries. We show that the proposed method iteratively handles differences in structure between depth and intensity images. This property enables reducing texture copying and depth bleeding artifacts significantly on a variety of range data sets. We also propose a fast alternating direction method of multipliers algorithm to solve our optimization problem. Our solver shows a noticeable speed up compared with the conventional majorize-minimize algorithm. Extensive experiments with synthetic and real-world data sets demonstrate that the proposed method is superior to the existing methods.",
author = "Youngjung Kim and Bumsub Ham and Changjae Oh and Kwanghoon Sohn",
year = "2016",
month = "11",
day = "1",
doi = "10.1109/TIP.2016.2601262",
language = "English",
volume = "25",
pages = "5227--5238",
journal = "IEEE Transactions on Image Processing",
issn = "1057-7149",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "11",

}

Structure Selective Depth Superresolution for RGB-D Cameras. / Kim, Youngjung; Ham, Bumsub; Oh, Changjae; Sohn, Kwanghoon.

In: IEEE Transactions on Image Processing, Vol. 25, No. 11, 7547322, 01.11.2016, p. 5227-5238.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Structure Selective Depth Superresolution for RGB-D Cameras

AU - Kim, Youngjung

AU - Ham, Bumsub

AU - Oh, Changjae

AU - Sohn, Kwanghoon

PY - 2016/11/1

Y1 - 2016/11/1

N2 - This paper describes a method for high-quality depth superresolution. The standard formulations of image-guided depth upsampling, using simple joint filtering or quadratic optimization, lead to texture copying and depth bleeding artifacts. These artifacts are caused by inherent discrepancy of structures in data from different sensors. Although there exists some correlation between depth and intensity discontinuities, they are different in distribution and formation. To tackle this problem, we formulate an optimization model using a nonconvex regularizer. A nonlocal affinity established in a high-dimensional feature space is used to offer precisely localized depth boundaries. We show that the proposed method iteratively handles differences in structure between depth and intensity images. This property enables reducing texture copying and depth bleeding artifacts significantly on a variety of range data sets. We also propose a fast alternating direction method of multipliers algorithm to solve our optimization problem. Our solver shows a noticeable speed up compared with the conventional majorize-minimize algorithm. Extensive experiments with synthetic and real-world data sets demonstrate that the proposed method is superior to the existing methods.

AB - This paper describes a method for high-quality depth superresolution. The standard formulations of image-guided depth upsampling, using simple joint filtering or quadratic optimization, lead to texture copying and depth bleeding artifacts. These artifacts are caused by inherent discrepancy of structures in data from different sensors. Although there exists some correlation between depth and intensity discontinuities, they are different in distribution and formation. To tackle this problem, we formulate an optimization model using a nonconvex regularizer. A nonlocal affinity established in a high-dimensional feature space is used to offer precisely localized depth boundaries. We show that the proposed method iteratively handles differences in structure between depth and intensity images. This property enables reducing texture copying and depth bleeding artifacts significantly on a variety of range data sets. We also propose a fast alternating direction method of multipliers algorithm to solve our optimization problem. Our solver shows a noticeable speed up compared with the conventional majorize-minimize algorithm. Extensive experiments with synthetic and real-world data sets demonstrate that the proposed method is superior to the existing methods.

UR - http://www.scopus.com/inward/record.url?scp=84991112137&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84991112137&partnerID=8YFLogxK

U2 - 10.1109/TIP.2016.2601262

DO - 10.1109/TIP.2016.2601262

M3 - Article

AN - SCOPUS:84991112137

VL - 25

SP - 5227

EP - 5238

JO - IEEE Transactions on Image Processing

JF - IEEE Transactions on Image Processing

SN - 1057-7149

IS - 11

M1 - 7547322

ER -