Automatic 2D-to-3D conversion using multi-scale deep neural network

Jiyoung Lee, Hyungjoo Jung, Youngjung Kim, Kwanghoon Sohn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We present a multi-scale deep convolutional neural network (CNN) for the task of automatic 2D-to-3D conversion. Traditional methods, which make a virtual view from a reference view, consist of separate stages i.e., depth (or disparity) estimation for the reference image and depth image-based rendering (DIBR) with estimated depth. In contrast, we reformulate the view synthesis task as an image reconstruction problem with a spatial transformer module and directly make stereo image pairs with a unified CNN framework without ground-truth depth as a supervision. We further propose a multi-scale deep architecture to capture the large displacements between images from coarse-level and enhance the detail from fine-level. Experimental results demonstrate the effectiveness of the proposed method over state-of-the-art approaches both qualitatively and quantitatively on the KITTI driving dataset.

Original languageEnglish
Title of host publication2017 IEEE International Conference on Image Processing, ICIP 2017 - Proceedings
PublisherIEEE Computer Society
Pages730-734
Number of pages5
ISBN (Electronic)9781509021758
DOIs
Publication statusPublished - 2018 Feb 20
Event24th IEEE International Conference on Image Processing, ICIP 2017 - Beijing, China
Duration: 2017 Sep 172017 Sep 20

Publication series

NameProceedings - International Conference on Image Processing, ICIP
Volume2017-September
ISSN (Print)1522-4880

Other

Other24th IEEE International Conference on Image Processing, ICIP 2017
CountryChina
CityBeijing
Period17/9/1717/9/20

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition
  • Signal Processing

Fingerprint Dive into the research topics of 'Automatic 2D-to-3D conversion using multi-scale deep neural network'. Together they form a unique fingerprint.

  • Cite this

    Lee, J., Jung, H., Kim, Y., & Sohn, K. (2018). Automatic 2D-to-3D conversion using multi-scale deep neural network. In 2017 IEEE International Conference on Image Processing, ICIP 2017 - Proceedings (pp. 730-734). (Proceedings - International Conference on Image Processing, ICIP; Vol. 2017-September). IEEE Computer Society. https://doi.org/10.1109/ICIP.2017.8296377