Unified confidence estimation networks for robust stereo matching

Sunok Kim, Dongbo Min, Seungryong Kim, Kwanghoon Sohn

Research output: Contribution to journalArticlepeer-review

20 Citations (Scopus)

Abstract

We present a deep architecture that estimates a stereo confidence, which is essential for improving the accuracy of stereo matching algorithms. In contrast to existing methods based on deep convolutional neural networks (CNNs) that rely on only one of the matching cost volume or estimated disparity map, our network estimates the stereo confidence by using the two heterogeneous inputs simultaneously. Specifically, the matching probability volume is first computed from the matching cost volume with residual networks and a pooling module in a manner that yields greater robustness. The confidence is then estimated through a unified deep network that combines confidence features extracted both from the matching probability volume and its corresponding disparity. In addition, our method extracts the confidence features of the disparity map by applying multiple convolutional filters with varying sizes to an input disparity map. To learn our networks in a semi-supervised manner, we propose a novel loss function that use confident points to compute the image reconstruction loss. To validate the effectiveness of our method in a disparity post-processing step, we employ three post-processing approaches; cost modulation, ground control points-based propagation, and aggregated ground control points-based propagation. Experimental results demonstrate that our method outperforms state-of-the-art confidence estimation methods on various benchmarks.

Original languageEnglish
Article number8510870
Pages (from-to)1299-1313
Number of pages15
JournalIEEE Transactions on Image Processing
Volume28
Issue number3
DOIs
Publication statusPublished - 2019 Mar

Bibliographical note

Funding Information:
Manuscript received February 9, 2018; revised July 13, 2018 and September 3, 2018; accepted October 21, 2018. Date of publication October 26, 2018; date of current version November 7, 2018. This work was supported by the Next-Generation Information Computing Development Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT under Grant NRF-2017M3C4A7069370. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Ce Zhu. (Corresponding author: Kwanghoon Sohn.) S. Kim, S. Kim, and K. Sohn are with the School of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, South Korea (e-mail: kso428@yonsei.ac.kr; srkim89@yonsei.ac.kr; khsohn@yonsei.ac.kr).

Publisher Copyright:
© 1992-2012 IEEE.

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Graphics and Computer-Aided Design

Fingerprint

Dive into the research topics of 'Unified confidence estimation networks for robust stereo matching'. Together they form a unique fingerprint.

Cite this