Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks

Seongjong Song, Hyunjung Shim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We propose a novel approach to recovering translucent objects from a single time-of-flight (ToF) depth camera using deep residual networks. When recording translucent objects using the ToF depth camera, their depth values are severely contaminated due to complex light interactions with surrounding environment. While existing methods suggested new capture systems or developed the depth distortion models, their solutions were less practical because of strict assumptions or heavy computational complexity. In this paper, we adopt deep residual networks for modeling the ToF depth distortion caused by translucency. To fully utilize both the local and semantic information of objects, multi-scale patches are used to predict the depth value. Based on the quantitative and qualitative evaluation on our benchmark database, we show the effectiveness and robustness of the proposed algorithm.

Original languageEnglish
Title of host publicationComputer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers
EditorsHongdong Li, Konrad Schindler, C.V. Jawahar, Greg Mori
PublisherSpringer Verlag
Pages641-657
Number of pages17
ISBN (Print)9783030208721
DOIs
Publication statusPublished - 2019 Jan 1
Event14th Asian Conference on Computer Vision, ACCV 2018 - Perth, Australia
Duration: 2018 Dec 22018 Dec 6

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11365 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference14th Asian Conference on Computer Vision, ACCV 2018
CountryAustralia
CityPerth
Period18/12/218/12/6

Fingerprint

Time-of-flight
Camera
Cameras
Computational complexity
Semantics
Patch
Object
Computational Complexity
Benchmark
Robustness
Predict
Evaluation
Interaction
Modeling

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Song, S., & Shim, H. (2019). Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks. In H. Li, K. Schindler, C. V. Jawahar, & G. Mori (Eds.), Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers (pp. 641-657). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11365 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-20873-8_41
Song, Seongjong ; Shim, Hyunjung. / Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks. Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. editor / Hongdong Li ; Konrad Schindler ; C.V. Jawahar ; Greg Mori. Springer Verlag, 2019. pp. 641-657 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{9830ecf71f284c23b1995f7782f133b7,
title = "Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks",
abstract = "We propose a novel approach to recovering translucent objects from a single time-of-flight (ToF) depth camera using deep residual networks. When recording translucent objects using the ToF depth camera, their depth values are severely contaminated due to complex light interactions with surrounding environment. While existing methods suggested new capture systems or developed the depth distortion models, their solutions were less practical because of strict assumptions or heavy computational complexity. In this paper, we adopt deep residual networks for modeling the ToF depth distortion caused by translucency. To fully utilize both the local and semantic information of objects, multi-scale patches are used to predict the depth value. Based on the quantitative and qualitative evaluation on our benchmark database, we show the effectiveness and robustness of the proposed algorithm.",
author = "Seongjong Song and Hyunjung Shim",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/978-3-030-20873-8_41",
language = "English",
isbn = "9783030208721",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "641--657",
editor = "Hongdong Li and Konrad Schindler and C.V. Jawahar and Greg Mori",
booktitle = "Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers",
address = "Germany",

}

Song, S & Shim, H 2019, Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks. in H Li, K Schindler, CV Jawahar & G Mori (eds), Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11365 LNCS, Springer Verlag, pp. 641-657, 14th Asian Conference on Computer Vision, ACCV 2018, Perth, Australia, 18/12/2. https://doi.org/10.1007/978-3-030-20873-8_41

Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks. / Song, Seongjong; Shim, Hyunjung.

Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. ed. / Hongdong Li; Konrad Schindler; C.V. Jawahar; Greg Mori. Springer Verlag, 2019. p. 641-657 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11365 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks

AU - Song, Seongjong

AU - Shim, Hyunjung

PY - 2019/1/1

Y1 - 2019/1/1

N2 - We propose a novel approach to recovering translucent objects from a single time-of-flight (ToF) depth camera using deep residual networks. When recording translucent objects using the ToF depth camera, their depth values are severely contaminated due to complex light interactions with surrounding environment. While existing methods suggested new capture systems or developed the depth distortion models, their solutions were less practical because of strict assumptions or heavy computational complexity. In this paper, we adopt deep residual networks for modeling the ToF depth distortion caused by translucency. To fully utilize both the local and semantic information of objects, multi-scale patches are used to predict the depth value. Based on the quantitative and qualitative evaluation on our benchmark database, we show the effectiveness and robustness of the proposed algorithm.

AB - We propose a novel approach to recovering translucent objects from a single time-of-flight (ToF) depth camera using deep residual networks. When recording translucent objects using the ToF depth camera, their depth values are severely contaminated due to complex light interactions with surrounding environment. While existing methods suggested new capture systems or developed the depth distortion models, their solutions were less practical because of strict assumptions or heavy computational complexity. In this paper, we adopt deep residual networks for modeling the ToF depth distortion caused by translucency. To fully utilize both the local and semantic information of objects, multi-scale patches are used to predict the depth value. Based on the quantitative and qualitative evaluation on our benchmark database, we show the effectiveness and robustness of the proposed algorithm.

UR - http://www.scopus.com/inward/record.url?scp=85066789281&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066789281&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-20873-8_41

DO - 10.1007/978-3-030-20873-8_41

M3 - Conference contribution

SN - 9783030208721

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 641

EP - 657

BT - Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers

A2 - Li, Hongdong

A2 - Schindler, Konrad

A2 - Jawahar, C.V.

A2 - Mori, Greg

PB - Springer Verlag

ER -

Song S, Shim H. Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera Using Deep Residual Networks. In Li H, Schindler K, Jawahar CV, Mori G, editors, Computer Vision – ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers. Springer Verlag. 2019. p. 641-657. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-20873-8_41