High-Precision Depth Estimation Using Uncalibrated LiDAR and Stereo Fusion

Kihong Park, Seungryong Kim, Kwanghoon Sohn

Research output: Contribution to journalArticlepeer-review

10 Citations (Scopus)

Abstract

We address the problem of 3D reconstruction from uncalibrated LiDAR point cloud and stereo images. Since the usage of each sensor alone for 3D reconstruction has weaknesses in terms of density and accuracy, we propose a deep sensor fusion framework for high-precision depth estimation. The proposed architecture consists of calibration network and depth fusion network, where both networks are designed considering the trade-off between accuracy and efficiency for mobile devices. The calibration network first corrects an initial extrinsic parameter to align the input sensor coordinate systems. The accuracy of calibration is markedly improved by formulating the calibration in the depth domain. In the depth fusion network, complementary characteristics of sparse LiDAR and dense stereo depth are then encoded in a boosting manner. Since training data for the LiDAR and stereo depth fusion are rather limited, we introduce a simple but effective approach to generate pseudo ground truth labels from the raw KITTI dataset. The experimental evaluation verifies that the proposed method outperforms current state-of-the-art methods on the KITTI benchmark. We also collect data using our proprietary multi-sensor acquisition platform and verify that the proposed method generalizes across different sensor settings and scenes.

Original languageEnglish
Article number8624583
Pages (from-to)321-335
Number of pages15
JournalIEEE Transactions on Intelligent Transportation Systems
Volume21
Issue number1
DOIs
Publication statusPublished - 2020 Jan

Bibliographical note

Funding Information:
This work was supported by the Institute for Information and Communications Technology Promotion (IITP) funded by the Korean Government (MSIP) under Grant 2016-0-00197.

Funding Information:
Manuscript received June 1, 2018; revised November 16, 2018; accepted December 21, 2018. Date of publication January 23, 2019; date of current version December 31, 2019. This work was supported by the Institute for Information and Communications Technology Promotion (IITP) funded by the Korean Government (MSIP) under Grant 2016-0-00197. The Associate Editor for this paper was H. G. Jung. (Corresponding author: Kwanghoon Sohn.) The authors are with the School of Electrical and Electronic Engineering, Yonsei University, Seoul 120-749, South Korea (e-mail: khpark7727@yonsei.ac.kr; srkim89@yonsei.ac.kr; khsohn@yonsei.ac.kr). Digital Object Identifier 10.1109/TITS.2019.2891788

Publisher Copyright:
© 2000-2011 IEEE.

All Science Journal Classification (ASJC) codes

  • Automotive Engineering
  • Mechanical Engineering
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'High-Precision Depth Estimation Using Uncalibrated LiDAR and Stereo Fusion'. Together they form a unique fingerprint.

Cite this