High-precision depth estimation with the 3D LiDAR and stereo fusion

Kihong Park, Seungryong Kim, Kwanghoon Sohn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

We present a deep convolutional neural network (CNN) architecture for high-precision depth estimation by jointly utilizing sparse 3D LiDAR and dense stereo depth information. In this network, the complementary characteristics of sparse 3D LiDAR and dense stereo depth are simultaneously encoded in a boosting manner. Tailored to the LiDAR and stereo fusion problem, the proposed network differs from previous CNNs in the incorporation of a compact convolution module, which can be deployed with the constraints of mobile devices. As training data for the LiDAR and stereo fusion is rather limited, we introduce a simple yet effective approach for reproducing the raw KITTI dataset. The raw LiDAR scans are augmented by adapting an off-the-shelf stereo algorithm and a confidence measure. We evaluate the proposed network on the KITTI benchmark and data collected by our multi-sensor acquisition system. Experiments demonstrate that the proposed network generalizes across datasets and is significantly more accurate than various baseline approaches.

Original languageEnglish
Title of host publication2018 IEEE International Conference on Robotics and Automation, ICRA 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2156-2163
Number of pages8
ISBN (Electronic)9781538630815
DOIs
Publication statusPublished - 2018 Sep 10
Event2018 IEEE International Conference on Robotics and Automation, ICRA 2018 - Brisbane, Australia
Duration: 2018 May 212018 May 25

Publication series

NameProceedings - IEEE International Conference on Robotics and Automation
ISSN (Print)1050-4729

Conference

Conference2018 IEEE International Conference on Robotics and Automation, ICRA 2018
CountryAustralia
CityBrisbane
Period18/5/2118/5/25

Fingerprint

Fusion reactions
Network architecture
Convolution
Mobile devices
Neural networks
Sensors
Experiments

All Science Journal Classification (ASJC) codes

  • Software
  • Control and Systems Engineering
  • Artificial Intelligence
  • Electrical and Electronic Engineering

Cite this

Park, K., Kim, S., & Sohn, K. (2018). High-precision depth estimation with the 3D LiDAR and stereo fusion. In 2018 IEEE International Conference on Robotics and Automation, ICRA 2018 (pp. 2156-2163). [8461048] (Proceedings - IEEE International Conference on Robotics and Automation). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICRA.2018.8461048
Park, Kihong ; Kim, Seungryong ; Sohn, Kwanghoon. / High-precision depth estimation with the 3D LiDAR and stereo fusion. 2018 IEEE International Conference on Robotics and Automation, ICRA 2018. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 2156-2163 (Proceedings - IEEE International Conference on Robotics and Automation).
@inproceedings{71ac3f2ad5cf42f4a960f9eac9c58050,
title = "High-precision depth estimation with the 3D LiDAR and stereo fusion",
abstract = "We present a deep convolutional neural network (CNN) architecture for high-precision depth estimation by jointly utilizing sparse 3D LiDAR and dense stereo depth information. In this network, the complementary characteristics of sparse 3D LiDAR and dense stereo depth are simultaneously encoded in a boosting manner. Tailored to the LiDAR and stereo fusion problem, the proposed network differs from previous CNNs in the incorporation of a compact convolution module, which can be deployed with the constraints of mobile devices. As training data for the LiDAR and stereo fusion is rather limited, we introduce a simple yet effective approach for reproducing the raw KITTI dataset. The raw LiDAR scans are augmented by adapting an off-the-shelf stereo algorithm and a confidence measure. We evaluate the proposed network on the KITTI benchmark and data collected by our multi-sensor acquisition system. Experiments demonstrate that the proposed network generalizes across datasets and is significantly more accurate than various baseline approaches.",
author = "Kihong Park and Seungryong Kim and Kwanghoon Sohn",
year = "2018",
month = "9",
day = "10",
doi = "10.1109/ICRA.2018.8461048",
language = "English",
series = "Proceedings - IEEE International Conference on Robotics and Automation",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "2156--2163",
booktitle = "2018 IEEE International Conference on Robotics and Automation, ICRA 2018",
address = "United States",

}

Park, K, Kim, S & Sohn, K 2018, High-precision depth estimation with the 3D LiDAR and stereo fusion. in 2018 IEEE International Conference on Robotics and Automation, ICRA 2018., 8461048, Proceedings - IEEE International Conference on Robotics and Automation, Institute of Electrical and Electronics Engineers Inc., pp. 2156-2163, 2018 IEEE International Conference on Robotics and Automation, ICRA 2018, Brisbane, Australia, 18/5/21. https://doi.org/10.1109/ICRA.2018.8461048

High-precision depth estimation with the 3D LiDAR and stereo fusion. / Park, Kihong; Kim, Seungryong; Sohn, Kwanghoon.

2018 IEEE International Conference on Robotics and Automation, ICRA 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 2156-2163 8461048 (Proceedings - IEEE International Conference on Robotics and Automation).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - High-precision depth estimation with the 3D LiDAR and stereo fusion

AU - Park, Kihong

AU - Kim, Seungryong

AU - Sohn, Kwanghoon

PY - 2018/9/10

Y1 - 2018/9/10

N2 - We present a deep convolutional neural network (CNN) architecture for high-precision depth estimation by jointly utilizing sparse 3D LiDAR and dense stereo depth information. In this network, the complementary characteristics of sparse 3D LiDAR and dense stereo depth are simultaneously encoded in a boosting manner. Tailored to the LiDAR and stereo fusion problem, the proposed network differs from previous CNNs in the incorporation of a compact convolution module, which can be deployed with the constraints of mobile devices. As training data for the LiDAR and stereo fusion is rather limited, we introduce a simple yet effective approach for reproducing the raw KITTI dataset. The raw LiDAR scans are augmented by adapting an off-the-shelf stereo algorithm and a confidence measure. We evaluate the proposed network on the KITTI benchmark and data collected by our multi-sensor acquisition system. Experiments demonstrate that the proposed network generalizes across datasets and is significantly more accurate than various baseline approaches.

AB - We present a deep convolutional neural network (CNN) architecture for high-precision depth estimation by jointly utilizing sparse 3D LiDAR and dense stereo depth information. In this network, the complementary characteristics of sparse 3D LiDAR and dense stereo depth are simultaneously encoded in a boosting manner. Tailored to the LiDAR and stereo fusion problem, the proposed network differs from previous CNNs in the incorporation of a compact convolution module, which can be deployed with the constraints of mobile devices. As training data for the LiDAR and stereo fusion is rather limited, we introduce a simple yet effective approach for reproducing the raw KITTI dataset. The raw LiDAR scans are augmented by adapting an off-the-shelf stereo algorithm and a confidence measure. We evaluate the proposed network on the KITTI benchmark and data collected by our multi-sensor acquisition system. Experiments demonstrate that the proposed network generalizes across datasets and is significantly more accurate than various baseline approaches.

UR - http://www.scopus.com/inward/record.url?scp=85063136324&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063136324&partnerID=8YFLogxK

U2 - 10.1109/ICRA.2018.8461048

DO - 10.1109/ICRA.2018.8461048

M3 - Conference contribution

AN - SCOPUS:85063136324

T3 - Proceedings - IEEE International Conference on Robotics and Automation

SP - 2156

EP - 2163

BT - 2018 IEEE International Conference on Robotics and Automation, ICRA 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Park K, Kim S, Sohn K. High-precision depth estimation with the 3D LiDAR and stereo fusion. In 2018 IEEE International Conference on Robotics and Automation, ICRA 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 2156-2163. 8461048. (Proceedings - IEEE International Conference on Robotics and Automation). https://doi.org/10.1109/ICRA.2018.8461048