A study on the effects of RGB-D database scale and quality on depth analogy performance

Sunok Kim, Youngjung Kim, Kwanghoon Sohn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

In the past few years, depth estimation from a single image has received increased attentions due to its wide applicability in image and video understanding. For realizing these tasks, many approaches have been developed for estimating depth from a single image based on various depth cues such as shading, motion, etc. However, they failed to estimate plausible depth map when input color image is derived from different category in training images. To alleviate these problems, data-driven approaches have been popularly developed by leveraging the discriminative power of a large scale RGB-D database. These approaches assume that there exists appearance- depth correlation in natural scenes. However, this assumption is likely to be ambiguous when local image regions have similar appearance but different geometric placement within the scene. Recently, a depth analogy (DA) has been developed by using the correlation between color image and depth gradient. DA addresses depth ambiguity problem effectively and shows reliable performance. However, no experiments are conducted to investigate the relationship between database scale and the quality of the estimated depth map. In this paper, we extensively examine the effects of database scale and quality on the performance of DA method. In order to compare the quality of DA, we collect a large scale RGB-D database using Microsoft Kinect v1 and Kinect v2 on indoor and ZED stereo camera on outdoor environments. Since the depth map obtained by Kinect v2 has high quality compared to that of Kinect v1, the depth maps from the database from Kinect v2 are more reliable. It represents that the high quality and large scale RGB-D database guarantees the high quality of the depth estimation. The experimental results show that the high quality and large scale training database leads high quality estimated depth map in both indoor and outdoor scenes.

Original languageEnglish
Title of host publicationThree-Dimensional Imaging, Visualization, and Display 2016
EditorsBahram Javidi, Jung-Young Son
PublisherSPIE
ISBN (Electronic)9781510601086
DOIs
Publication statusPublished - 2016 Jan 1
EventThree-Dimensional Imaging, Visualization, and Display 2016 Conference - Baltimore, United States
Duration: 2016 Apr 182016 Apr 20

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume9867
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X

Other

OtherThree-Dimensional Imaging, Visualization, and Display 2016 Conference
CountryUnited States
CityBaltimore
Period16/4/1816/4/20

Fingerprint

Analogy
Depth Map
Depth Estimation
Color Image
Color
Shading
Ambiguous
Data-driven
Cameras
Placement
education
Camera
Likely
color
Gradient
cues
Motion
Experimental Results
ambiguity
Experiments

All Science Journal Classification (ASJC) codes

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Cite this

Kim, S., Kim, Y., & Sohn, K. (2016). A study on the effects of RGB-D database scale and quality on depth analogy performance. In B. Javidi, & J-Y. Son (Eds.), Three-Dimensional Imaging, Visualization, and Display 2016 [986707] (Proceedings of SPIE - The International Society for Optical Engineering; Vol. 9867). SPIE. https://doi.org/10.1117/12.2229600
Kim, Sunok ; Kim, Youngjung ; Sohn, Kwanghoon. / A study on the effects of RGB-D database scale and quality on depth analogy performance. Three-Dimensional Imaging, Visualization, and Display 2016. editor / Bahram Javidi ; Jung-Young Son. SPIE, 2016. (Proceedings of SPIE - The International Society for Optical Engineering).
@inproceedings{86291c3394d04553a4b12d2611342b87,
title = "A study on the effects of RGB-D database scale and quality on depth analogy performance",
abstract = "In the past few years, depth estimation from a single image has received increased attentions due to its wide applicability in image and video understanding. For realizing these tasks, many approaches have been developed for estimating depth from a single image based on various depth cues such as shading, motion, etc. However, they failed to estimate plausible depth map when input color image is derived from different category in training images. To alleviate these problems, data-driven approaches have been popularly developed by leveraging the discriminative power of a large scale RGB-D database. These approaches assume that there exists appearance- depth correlation in natural scenes. However, this assumption is likely to be ambiguous when local image regions have similar appearance but different geometric placement within the scene. Recently, a depth analogy (DA) has been developed by using the correlation between color image and depth gradient. DA addresses depth ambiguity problem effectively and shows reliable performance. However, no experiments are conducted to investigate the relationship between database scale and the quality of the estimated depth map. In this paper, we extensively examine the effects of database scale and quality on the performance of DA method. In order to compare the quality of DA, we collect a large scale RGB-D database using Microsoft Kinect v1 and Kinect v2 on indoor and ZED stereo camera on outdoor environments. Since the depth map obtained by Kinect v2 has high quality compared to that of Kinect v1, the depth maps from the database from Kinect v2 are more reliable. It represents that the high quality and large scale RGB-D database guarantees the high quality of the depth estimation. The experimental results show that the high quality and large scale training database leads high quality estimated depth map in both indoor and outdoor scenes.",
author = "Sunok Kim and Youngjung Kim and Kwanghoon Sohn",
year = "2016",
month = "1",
day = "1",
doi = "10.1117/12.2229600",
language = "English",
series = "Proceedings of SPIE - The International Society for Optical Engineering",
publisher = "SPIE",
editor = "Bahram Javidi and Jung-Young Son",
booktitle = "Three-Dimensional Imaging, Visualization, and Display 2016",
address = "United States",

}

Kim, S, Kim, Y & Sohn, K 2016, A study on the effects of RGB-D database scale and quality on depth analogy performance. in B Javidi & J-Y Son (eds), Three-Dimensional Imaging, Visualization, and Display 2016., 986707, Proceedings of SPIE - The International Society for Optical Engineering, vol. 9867, SPIE, Three-Dimensional Imaging, Visualization, and Display 2016 Conference, Baltimore, United States, 16/4/18. https://doi.org/10.1117/12.2229600

A study on the effects of RGB-D database scale and quality on depth analogy performance. / Kim, Sunok; Kim, Youngjung; Sohn, Kwanghoon.

Three-Dimensional Imaging, Visualization, and Display 2016. ed. / Bahram Javidi; Jung-Young Son. SPIE, 2016. 986707 (Proceedings of SPIE - The International Society for Optical Engineering; Vol. 9867).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - A study on the effects of RGB-D database scale and quality on depth analogy performance

AU - Kim, Sunok

AU - Kim, Youngjung

AU - Sohn, Kwanghoon

PY - 2016/1/1

Y1 - 2016/1/1

N2 - In the past few years, depth estimation from a single image has received increased attentions due to its wide applicability in image and video understanding. For realizing these tasks, many approaches have been developed for estimating depth from a single image based on various depth cues such as shading, motion, etc. However, they failed to estimate plausible depth map when input color image is derived from different category in training images. To alleviate these problems, data-driven approaches have been popularly developed by leveraging the discriminative power of a large scale RGB-D database. These approaches assume that there exists appearance- depth correlation in natural scenes. However, this assumption is likely to be ambiguous when local image regions have similar appearance but different geometric placement within the scene. Recently, a depth analogy (DA) has been developed by using the correlation between color image and depth gradient. DA addresses depth ambiguity problem effectively and shows reliable performance. However, no experiments are conducted to investigate the relationship between database scale and the quality of the estimated depth map. In this paper, we extensively examine the effects of database scale and quality on the performance of DA method. In order to compare the quality of DA, we collect a large scale RGB-D database using Microsoft Kinect v1 and Kinect v2 on indoor and ZED stereo camera on outdoor environments. Since the depth map obtained by Kinect v2 has high quality compared to that of Kinect v1, the depth maps from the database from Kinect v2 are more reliable. It represents that the high quality and large scale RGB-D database guarantees the high quality of the depth estimation. The experimental results show that the high quality and large scale training database leads high quality estimated depth map in both indoor and outdoor scenes.

AB - In the past few years, depth estimation from a single image has received increased attentions due to its wide applicability in image and video understanding. For realizing these tasks, many approaches have been developed for estimating depth from a single image based on various depth cues such as shading, motion, etc. However, they failed to estimate plausible depth map when input color image is derived from different category in training images. To alleviate these problems, data-driven approaches have been popularly developed by leveraging the discriminative power of a large scale RGB-D database. These approaches assume that there exists appearance- depth correlation in natural scenes. However, this assumption is likely to be ambiguous when local image regions have similar appearance but different geometric placement within the scene. Recently, a depth analogy (DA) has been developed by using the correlation between color image and depth gradient. DA addresses depth ambiguity problem effectively and shows reliable performance. However, no experiments are conducted to investigate the relationship between database scale and the quality of the estimated depth map. In this paper, we extensively examine the effects of database scale and quality on the performance of DA method. In order to compare the quality of DA, we collect a large scale RGB-D database using Microsoft Kinect v1 and Kinect v2 on indoor and ZED stereo camera on outdoor environments. Since the depth map obtained by Kinect v2 has high quality compared to that of Kinect v1, the depth maps from the database from Kinect v2 are more reliable. It represents that the high quality and large scale RGB-D database guarantees the high quality of the depth estimation. The experimental results show that the high quality and large scale training database leads high quality estimated depth map in both indoor and outdoor scenes.

UR - http://www.scopus.com/inward/record.url?scp=84982273825&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84982273825&partnerID=8YFLogxK

U2 - 10.1117/12.2229600

DO - 10.1117/12.2229600

M3 - Conference contribution

T3 - Proceedings of SPIE - The International Society for Optical Engineering

BT - Three-Dimensional Imaging, Visualization, and Display 2016

A2 - Javidi, Bahram

A2 - Son, Jung-Young

PB - SPIE

ER -

Kim S, Kim Y, Sohn K. A study on the effects of RGB-D database scale and quality on depth analogy performance. In Javidi B, Son J-Y, editors, Three-Dimensional Imaging, Visualization, and Display 2016. SPIE. 2016. 986707. (Proceedings of SPIE - The International Society for Optical Engineering). https://doi.org/10.1117/12.2229600