Multi-view tof sensor fusion technique for high quality depth map

Deukhyeon Kim, Jinwook Choi, Kwanghoon Sohn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

The Time-of-Flight (ToF) sensor has been widely used in computer vision fields since it can provide depth information in real time. However, the depth map obtained from ToF sensor is distressed with error, and has a lower resolution than general cameras. In this paper, we propose a novel framework to fuse and upsample multi-view depth maps obtained from multiple ToF sensors. The proposed method can be robust to the camera calibration error and effectively applied to the Multi-view Video plus Depth (MVD) system. For that, we perform depth balancing and confidence map based multi-view depth fusion. The depth balancing adjusts the distribution of depth values between multiple ToF sensors. It can provide a coherent depth for the corresponding points between depth maps. Confidence map based multi-view depth fusion technique can restore the depth acquisition error and align multiple depth maps well with the corresponding color image by using only reliable depth values. Experimental results show that the proposed method using multiple ToF sensors is superior to the conventional method based on the 2D-plus-depth system consisting of one color camera and one depth sensor.

Original languageEnglish
Title of host publicationProceedings of SPIE-IS and T Electronic Imaging - Three-Dimensional Image Processing (3DIP) and Applications 2013
Volume8650
DOIs
Publication statusPublished - 2013 Jun 3
EventThree-Dimensional Image Processing (3DIP) and Applications 2013 - Burlingame, CA, United States
Duration: 2013 Feb 62013 Feb 7

Other

OtherThree-Dimensional Image Processing (3DIP) and Applications 2013
CountryUnited States
CityBurlingame, CA
Period13/2/613/2/7

Fingerprint

Sensor Fusion
Depth Map
multisensor fusion
Fusion reactions
Sensors
Time-of-flight
Cameras
Sensor
Color
sensors
Electric fuses
Balancing
Computer vision
Confidence
cameras
Fusion
Camera
Calibration
confidence
Camera Calibration

All Science Journal Classification (ASJC) codes

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Cite this

Kim, D., Choi, J., & Sohn, K. (2013). Multi-view tof sensor fusion technique for high quality depth map. In Proceedings of SPIE-IS and T Electronic Imaging - Three-Dimensional Image Processing (3DIP) and Applications 2013 (Vol. 8650). [865006] https://doi.org/10.1117/12.2001733
Kim, Deukhyeon ; Choi, Jinwook ; Sohn, Kwanghoon. / Multi-view tof sensor fusion technique for high quality depth map. Proceedings of SPIE-IS and T Electronic Imaging - Three-Dimensional Image Processing (3DIP) and Applications 2013. Vol. 8650 2013.
@inproceedings{38f4c4ed165b4240ba6a82c00996e1ff,
title = "Multi-view tof sensor fusion technique for high quality depth map",
abstract = "The Time-of-Flight (ToF) sensor has been widely used in computer vision fields since it can provide depth information in real time. However, the depth map obtained from ToF sensor is distressed with error, and has a lower resolution than general cameras. In this paper, we propose a novel framework to fuse and upsample multi-view depth maps obtained from multiple ToF sensors. The proposed method can be robust to the camera calibration error and effectively applied to the Multi-view Video plus Depth (MVD) system. For that, we perform depth balancing and confidence map based multi-view depth fusion. The depth balancing adjusts the distribution of depth values between multiple ToF sensors. It can provide a coherent depth for the corresponding points between depth maps. Confidence map based multi-view depth fusion technique can restore the depth acquisition error and align multiple depth maps well with the corresponding color image by using only reliable depth values. Experimental results show that the proposed method using multiple ToF sensors is superior to the conventional method based on the 2D-plus-depth system consisting of one color camera and one depth sensor.",
author = "Deukhyeon Kim and Jinwook Choi and Kwanghoon Sohn",
year = "2013",
month = "6",
day = "3",
doi = "10.1117/12.2001733",
language = "English",
isbn = "9780819494238",
volume = "8650",
booktitle = "Proceedings of SPIE-IS and T Electronic Imaging - Three-Dimensional Image Processing (3DIP) and Applications 2013",

}

Kim, D, Choi, J & Sohn, K 2013, Multi-view tof sensor fusion technique for high quality depth map. in Proceedings of SPIE-IS and T Electronic Imaging - Three-Dimensional Image Processing (3DIP) and Applications 2013. vol. 8650, 865006, Three-Dimensional Image Processing (3DIP) and Applications 2013, Burlingame, CA, United States, 13/2/6. https://doi.org/10.1117/12.2001733

Multi-view tof sensor fusion technique for high quality depth map. / Kim, Deukhyeon; Choi, Jinwook; Sohn, Kwanghoon.

Proceedings of SPIE-IS and T Electronic Imaging - Three-Dimensional Image Processing (3DIP) and Applications 2013. Vol. 8650 2013. 865006.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Multi-view tof sensor fusion technique for high quality depth map

AU - Kim, Deukhyeon

AU - Choi, Jinwook

AU - Sohn, Kwanghoon

PY - 2013/6/3

Y1 - 2013/6/3

N2 - The Time-of-Flight (ToF) sensor has been widely used in computer vision fields since it can provide depth information in real time. However, the depth map obtained from ToF sensor is distressed with error, and has a lower resolution than general cameras. In this paper, we propose a novel framework to fuse and upsample multi-view depth maps obtained from multiple ToF sensors. The proposed method can be robust to the camera calibration error and effectively applied to the Multi-view Video plus Depth (MVD) system. For that, we perform depth balancing and confidence map based multi-view depth fusion. The depth balancing adjusts the distribution of depth values between multiple ToF sensors. It can provide a coherent depth for the corresponding points between depth maps. Confidence map based multi-view depth fusion technique can restore the depth acquisition error and align multiple depth maps well with the corresponding color image by using only reliable depth values. Experimental results show that the proposed method using multiple ToF sensors is superior to the conventional method based on the 2D-plus-depth system consisting of one color camera and one depth sensor.

AB - The Time-of-Flight (ToF) sensor has been widely used in computer vision fields since it can provide depth information in real time. However, the depth map obtained from ToF sensor is distressed with error, and has a lower resolution than general cameras. In this paper, we propose a novel framework to fuse and upsample multi-view depth maps obtained from multiple ToF sensors. The proposed method can be robust to the camera calibration error and effectively applied to the Multi-view Video plus Depth (MVD) system. For that, we perform depth balancing and confidence map based multi-view depth fusion. The depth balancing adjusts the distribution of depth values between multiple ToF sensors. It can provide a coherent depth for the corresponding points between depth maps. Confidence map based multi-view depth fusion technique can restore the depth acquisition error and align multiple depth maps well with the corresponding color image by using only reliable depth values. Experimental results show that the proposed method using multiple ToF sensors is superior to the conventional method based on the 2D-plus-depth system consisting of one color camera and one depth sensor.

UR - http://www.scopus.com/inward/record.url?scp=84878275605&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84878275605&partnerID=8YFLogxK

U2 - 10.1117/12.2001733

DO - 10.1117/12.2001733

M3 - Conference contribution

SN - 9780819494238

VL - 8650

BT - Proceedings of SPIE-IS and T Electronic Imaging - Three-Dimensional Image Processing (3DIP) and Applications 2013

ER -

Kim D, Choi J, Sohn K. Multi-view tof sensor fusion technique for high quality depth map. In Proceedings of SPIE-IS and T Electronic Imaging - Three-Dimensional Image Processing (3DIP) and Applications 2013. Vol. 8650. 2013. 865006 https://doi.org/10.1117/12.2001733