Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields

Seungryong Kim; Kihong Park; Kwanghoon Sohn; Stephen Lin

doi:10.1007/978-3-319-46484-8_9

Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields

Seungryong Kim, Kihong Park, Kwanghoon Sohn, Stephen Lin

Department of Electrical and Electronic Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

44 Citations (Scopus)

Abstract

We present a method for jointly predicting a depth map and intrinsic images from single-image input. The two tasks are formulated in a synergistic manner through a joint conditional random field (CRF) that is solved using a novel convolutional neural network (CNN) architecture, called the joint convolutional neural field (JCNF) model. Tailored to our joint estimation problem, JCNF differs from previous CNNs in its sharing of convolutional activations and layers between networks for each task, its inference in the gradient domain where there exists greater correlation between depth and intrinsic images, and the incorporation of a gradient scale network that learns the confidence of estimated gradients in order to effectively balance them in the solution. This approach is shown to surpass state-of-the-art methods both on single-image depth estimation and on intrinsic image decomposition.

Original language	English
Title of host publication	Computer Vision - 14th European Conference, ECCV 2016, Proceedings
Editors	Bastian Leibe, Jiri Matas, Nicu Sebe, Max Welling
Publisher	Springer Verlag
Pages	143-159
Number of pages	17
ISBN (Print)	9783319464831
DOIs	https://doi.org/10.1007/978-3-319-46484-8_9
Publication status	Published - 2016
Event	14th European Conference on Computer Vision, ECCV 2016 - Amsterdam, Netherlands Duration: 2016 Oct 8 → 2016 Oct 16

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9912 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	14th European Conference on Computer Vision, ECCV 2016
Country/Territory	Netherlands
City	Amsterdam
Period	16/10/8 → 16/10/16

Bibliographical note

Funding Information:
This research was supported by the MSIP (The Ministry of Science, ICT and Future Planning), Korea and Microsoft Research, under ICT/SW Creative research program supervised by the IITP(Institute for Information & Communications Technology Promotion) (IITP-2015-R2212-15-0008).

Publisher Copyright:
© Springer International Publishing AG 2016.

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-319-46484-8_9

Cite this

Kim, S., Park, K., Sohn, K., & Lin, S. (2016). Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields. In B. Leibe, J. Matas, N. Sebe, & M. Welling (Eds.), Computer Vision - 14th European Conference, ECCV 2016, Proceedings (pp. 143-159). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9912 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-46484-8_9

Kim, Seungryong ; Park, Kihong ; Sohn, Kwanghoon ; Lin, Stephen. / Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields. Computer Vision - 14th European Conference, ECCV 2016, Proceedings. editor / Bastian Leibe ; Jiri Matas ; Nicu Sebe ; Max Welling. Springer Verlag, 2016. pp. 143-159 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{e92611151d784536b2fb7e42c16c75dc,

title = "Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields",

abstract = "We present a method for jointly predicting a depth map and intrinsic images from single-image input. The two tasks are formulated in a synergistic manner through a joint conditional random field (CRF) that is solved using a novel convolutional neural network (CNN) architecture, called the joint convolutional neural field (JCNF) model. Tailored to our joint estimation problem, JCNF differs from previous CNNs in its sharing of convolutional activations and layers between networks for each task, its inference in the gradient domain where there exists greater correlation between depth and intrinsic images, and the incorporation of a gradient scale network that learns the confidence of estimated gradients in order to effectively balance them in the solution. This approach is shown to surpass state-of-the-art methods both on single-image depth estimation and on intrinsic image decomposition.",

author = "Seungryong Kim and Kihong Park and Kwanghoon Sohn and Stephen Lin",

note = "Funding Information: This research was supported by the MSIP (The Ministry of Science, ICT and Future Planning), Korea and Microsoft Research, under ICT/SW Creative research program supervised by the IITP(Institute for Information & Communications Technology Promotion) (IITP-2015-R2212-15-0008). Publisher Copyright: {\textcopyright} Springer International Publishing AG 2016.; 14th European Conference on Computer Vision, ECCV 2016 ; Conference date: 08-10-2016 Through 16-10-2016",

year = "2016",

doi = "10.1007/978-3-319-46484-8_9",

language = "English",

isbn = "9783319464831",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "143--159",

editor = "Bastian Leibe and Jiri Matas and Nicu Sebe and Max Welling",

booktitle = "Computer Vision - 14th European Conference, ECCV 2016, Proceedings",

address = "Germany",

}

Kim, S, Park, K, Sohn, K & Lin, S 2016, Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields. in B Leibe, J Matas, N Sebe & M Welling (eds), Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9912 LNCS, Springer Verlag, pp. 143-159, 14th European Conference on Computer Vision, ECCV 2016, Amsterdam, Netherlands, 16/10/8. https://doi.org/10.1007/978-3-319-46484-8_9

Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields. / Kim, Seungryong; Park, Kihong; Sohn, Kwanghoon; Lin, Stephen.

Computer Vision - 14th European Conference, ECCV 2016, Proceedings. ed. / Bastian Leibe; Jiri Matas; Nicu Sebe; Max Welling. Springer Verlag, 2016. p. 143-159 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9912 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields

AU - Kim, Seungryong

AU - Park, Kihong

AU - Sohn, Kwanghoon

AU - Lin, Stephen

N1 - Funding Information: This research was supported by the MSIP (The Ministry of Science, ICT and Future Planning), Korea and Microsoft Research, under ICT/SW Creative research program supervised by the IITP(Institute for Information & Communications Technology Promotion) (IITP-2015-R2212-15-0008). Publisher Copyright: © Springer International Publishing AG 2016.

PY - 2016

Y1 - 2016

N2 - We present a method for jointly predicting a depth map and intrinsic images from single-image input. The two tasks are formulated in a synergistic manner through a joint conditional random field (CRF) that is solved using a novel convolutional neural network (CNN) architecture, called the joint convolutional neural field (JCNF) model. Tailored to our joint estimation problem, JCNF differs from previous CNNs in its sharing of convolutional activations and layers between networks for each task, its inference in the gradient domain where there exists greater correlation between depth and intrinsic images, and the incorporation of a gradient scale network that learns the confidence of estimated gradients in order to effectively balance them in the solution. This approach is shown to surpass state-of-the-art methods both on single-image depth estimation and on intrinsic image decomposition.

AB - We present a method for jointly predicting a depth map and intrinsic images from single-image input. The two tasks are formulated in a synergistic manner through a joint conditional random field (CRF) that is solved using a novel convolutional neural network (CNN) architecture, called the joint convolutional neural field (JCNF) model. Tailored to our joint estimation problem, JCNF differs from previous CNNs in its sharing of convolutional activations and layers between networks for each task, its inference in the gradient domain where there exists greater correlation between depth and intrinsic images, and the incorporation of a gradient scale network that learns the confidence of estimated gradients in order to effectively balance them in the solution. This approach is shown to surpass state-of-the-art methods both on single-image depth estimation and on intrinsic image decomposition.

UR - http://www.scopus.com/inward/record.url?scp=84990060400&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84990060400&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-46484-8_9

DO - 10.1007/978-3-319-46484-8_9

M3 - Conference contribution

AN - SCOPUS:84990060400

SN - 9783319464831

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 143

EP - 159

BT - Computer Vision - 14th European Conference, ECCV 2016, Proceedings

A2 - Leibe, Bastian

A2 - Matas, Jiri

A2 - Sebe, Nicu

A2 - Welling, Max

PB - Springer Verlag

T2 - 14th European Conference on Computer Vision, ECCV 2016

Y2 - 8 October 2016 through 16 October 2016

ER -

Kim S, Park K, Sohn K, Lin S. Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields. In Leibe B, Matas J, Sebe N, Welling M, editors, Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Springer Verlag. 2016. p. 143-159. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-46484-8_9

Unified depth prediction and intrinsic image decomposition from a single image via joint convolutional neural fields

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this