Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields

Seungryong Kim, Kihong Park, Kwanghoon Sohn, Stephen Lin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

15 Citations (Scopus)

Abstract

We present a method for jointly predicting a depth map and intrinsic images from single-image input. The two tasks are formulated in a synergistic manner through a joint conditional random field (CRF) that is solved using a novel convolutional neural network (CNN) architecture, called the joint convolutional neural field (JCNF) model. Tailored to our joint estimation problem, JCNF differs from previous CNNs in its sharing of convolutional activations and layers between networks for each task, its inference in the gradient domain where there exists greater correlation between depth and intrinsic images, and the incorporation of a gradient scale network that learns the confidence of estimated gradients in order to effectively balance them in the solution. This approach is shown to surpass state-of-the-art methods both on single-image depth estimation and on intrinsic image decomposition.

Original languageEnglish
Title of host publicationComputer Vision - 14th European Conference, ECCV 2016, Proceedings
EditorsBastian Leibe, Jiri Matas, Max Welling, Nicu Sebe
PublisherSpringer Verlag
Pages143-159
Number of pages17
ISBN (Print)9783319464831
DOIs
Publication statusPublished - 2016 Jan 1

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9912 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Fingerprint

Image Decomposition
Decomposition
Network layers
Prediction
Gradient
Network architecture
Chemical activation
Depth Estimation
Neural networks
Conditional Random Fields
Depth Map
Network Architecture
Confidence
Activation
Sharing
Neural Networks

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Kim, S., Park, K., Sohn, K., & Lin, S. (2016). Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields. In B. Leibe, J. Matas, M. Welling, & N. Sebe (Eds.), Computer Vision - 14th European Conference, ECCV 2016, Proceedings (pp. 143-159). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9912 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-46484-8_9
Kim, Seungryong ; Park, Kihong ; Sohn, Kwanghoon ; Lin, Stephen. / Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields. Computer Vision - 14th European Conference, ECCV 2016, Proceedings. editor / Bastian Leibe ; Jiri Matas ; Max Welling ; Nicu Sebe. Springer Verlag, 2016. pp. 143-159 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{e92611151d784536b2fb7e42c16c75dc,
title = "Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields",
abstract = "We present a method for jointly predicting a depth map and intrinsic images from single-image input. The two tasks are formulated in a synergistic manner through a joint conditional random field (CRF) that is solved using a novel convolutional neural network (CNN) architecture, called the joint convolutional neural field (JCNF) model. Tailored to our joint estimation problem, JCNF differs from previous CNNs in its sharing of convolutional activations and layers between networks for each task, its inference in the gradient domain where there exists greater correlation between depth and intrinsic images, and the incorporation of a gradient scale network that learns the confidence of estimated gradients in order to effectively balance them in the solution. This approach is shown to surpass state-of-the-art methods both on single-image depth estimation and on intrinsic image decomposition.",
author = "Seungryong Kim and Kihong Park and Kwanghoon Sohn and Stephen Lin",
year = "2016",
month = "1",
day = "1",
doi = "10.1007/978-3-319-46484-8_9",
language = "English",
isbn = "9783319464831",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "143--159",
editor = "Bastian Leibe and Jiri Matas and Max Welling and Nicu Sebe",
booktitle = "Computer Vision - 14th European Conference, ECCV 2016, Proceedings",
address = "Germany",

}

Kim, S, Park, K, Sohn, K & Lin, S 2016, Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields. in B Leibe, J Matas, M Welling & N Sebe (eds), Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9912 LNCS, Springer Verlag, pp. 143-159. https://doi.org/10.1007/978-3-319-46484-8_9

Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields. / Kim, Seungryong; Park, Kihong; Sohn, Kwanghoon; Lin, Stephen.

Computer Vision - 14th European Conference, ECCV 2016, Proceedings. ed. / Bastian Leibe; Jiri Matas; Max Welling; Nicu Sebe. Springer Verlag, 2016. p. 143-159 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9912 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields

AU - Kim, Seungryong

AU - Park, Kihong

AU - Sohn, Kwanghoon

AU - Lin, Stephen

PY - 2016/1/1

Y1 - 2016/1/1

N2 - We present a method for jointly predicting a depth map and intrinsic images from single-image input. The two tasks are formulated in a synergistic manner through a joint conditional random field (CRF) that is solved using a novel convolutional neural network (CNN) architecture, called the joint convolutional neural field (JCNF) model. Tailored to our joint estimation problem, JCNF differs from previous CNNs in its sharing of convolutional activations and layers between networks for each task, its inference in the gradient domain where there exists greater correlation between depth and intrinsic images, and the incorporation of a gradient scale network that learns the confidence of estimated gradients in order to effectively balance them in the solution. This approach is shown to surpass state-of-the-art methods both on single-image depth estimation and on intrinsic image decomposition.

AB - We present a method for jointly predicting a depth map and intrinsic images from single-image input. The two tasks are formulated in a synergistic manner through a joint conditional random field (CRF) that is solved using a novel convolutional neural network (CNN) architecture, called the joint convolutional neural field (JCNF) model. Tailored to our joint estimation problem, JCNF differs from previous CNNs in its sharing of convolutional activations and layers between networks for each task, its inference in the gradient domain where there exists greater correlation between depth and intrinsic images, and the incorporation of a gradient scale network that learns the confidence of estimated gradients in order to effectively balance them in the solution. This approach is shown to surpass state-of-the-art methods both on single-image depth estimation and on intrinsic image decomposition.

UR - http://www.scopus.com/inward/record.url?scp=84990060400&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84990060400&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-46484-8_9

DO - 10.1007/978-3-319-46484-8_9

M3 - Conference contribution

SN - 9783319464831

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 143

EP - 159

BT - Computer Vision - 14th European Conference, ECCV 2016, Proceedings

A2 - Leibe, Bastian

A2 - Matas, Jiri

A2 - Welling, Max

A2 - Sebe, Nicu

PB - Springer Verlag

ER -

Kim S, Park K, Sohn K, Lin S. Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields. In Leibe B, Matas J, Welling M, Sebe N, editors, Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Springer Verlag. 2016. p. 143-159. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-46484-8_9