Automatic 2D-to-3D conversion using multi-scale deep neural network

Jiyoung Lee, Hyungjoo Jung, Youngjung Kim, Kwanghoon Sohn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

We present a multi-scale deep convolutional neural network (CNN) for the task of automatic 2D-to-3D conversion. Traditional methods, which make a virtual view from a reference view, consist of separate stages i.e., depth (or disparity) estimation for the reference image and depth image-based rendering (DIBR) with estimated depth. In contrast, we reformulate the view synthesis task as an image reconstruction problem with a spatial transformer module and directly make stereo image pairs with a unified CNN framework without ground-truth depth as a supervision. We further propose a multi-scale deep architecture to capture the large displacements between images from coarse-level and enhance the detail from fine-level. Experimental results demonstrate the effectiveness of the proposed method over state-of-the-art approaches both qualitatively and quantitatively on the KITTI driving dataset.

Original languageEnglish
Title of host publication2017 IEEE International Conference on Image Processing, ICIP 2017 - Proceedings
PublisherIEEE Computer Society
Pages730-734
Number of pages5
ISBN (Electronic)9781509021758
DOIs
Publication statusPublished - 2018 Feb 20
Event24th IEEE International Conference on Image Processing, ICIP 2017 - Beijing, China
Duration: 2017 Sep 172017 Sep 20

Publication series

NameProceedings - International Conference on Image Processing, ICIP
Volume2017-September
ISSN (Print)1522-4880

Other

Other24th IEEE International Conference on Image Processing, ICIP 2017
Country/TerritoryChina
CityBeijing
Period17/9/1717/9/20

Bibliographical note

Funding Information:
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (NRF-2016R1A2A2A05921659).

Publisher Copyright:
© 2017 IEEE.

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition
  • Signal Processing

Fingerprint

Dive into the research topics of 'Automatic 2D-to-3D conversion using multi-scale deep neural network'. Together they form a unique fingerprint.

Cite this