Unified confidence estimation networks for robust stereo matching

Sunok Kim, Dongbo Min, Seungryong Kim, Kwanghoon Sohn

Research output: Contribution to journalArticle

Abstract

We present a deep architecture that estimates a stereo confidence, which is essential for improving the accuracy of stereo matching algorithms. In contrast to existing methods based on deep convolutional neural networks (CNNs) that rely on only one of the matching cost volume or estimated disparity map, our network estimates the stereo confidence by using the two heterogeneous inputs simultaneously. Specifically, the matching probability volume is first computed from the matching cost volume with residual networks and a pooling module in a manner that yields greater robustness. The confidence is then estimated through a unified deep network that combines confidence features extracted both from the matching probability volume and its corresponding disparity. In addition, our method extracts the confidence features of the disparity map by applying multiple convolutional filters with varying sizes to an input disparity map. To learn our networks in a semi-supervised manner, we propose a novel loss function that use confident points to compute the image reconstruction loss. To validate the effectiveness of our method in a disparity post-processing step, we employ three post-processing approaches; cost modulation, ground control points-based propagation, and aggregated ground control points-based propagation. Experimental results demonstrate that our method outperforms state-of-the-art confidence estimation methods on various benchmarks.

Original languageEnglish
Article number8510870
Pages (from-to)1299-1313
Number of pages15
JournalIEEE Transactions on Image Processing
Volume28
Issue number3
DOIs
Publication statusPublished - 2019 Mar 1

Fingerprint

Costs
Processing
Image reconstruction
Modulation
Neural networks

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Graphics and Computer-Aided Design

Cite this

Kim, Sunok ; Min, Dongbo ; Kim, Seungryong ; Sohn, Kwanghoon. / Unified confidence estimation networks for robust stereo matching. In: IEEE Transactions on Image Processing. 2019 ; Vol. 28, No. 3. pp. 1299-1313.
@article{0c938658af4f41daa0296d0273a44c44,
title = "Unified confidence estimation networks for robust stereo matching",
abstract = "We present a deep architecture that estimates a stereo confidence, which is essential for improving the accuracy of stereo matching algorithms. In contrast to existing methods based on deep convolutional neural networks (CNNs) that rely on only one of the matching cost volume or estimated disparity map, our network estimates the stereo confidence by using the two heterogeneous inputs simultaneously. Specifically, the matching probability volume is first computed from the matching cost volume with residual networks and a pooling module in a manner that yields greater robustness. The confidence is then estimated through a unified deep network that combines confidence features extracted both from the matching probability volume and its corresponding disparity. In addition, our method extracts the confidence features of the disparity map by applying multiple convolutional filters with varying sizes to an input disparity map. To learn our networks in a semi-supervised manner, we propose a novel loss function that use confident points to compute the image reconstruction loss. To validate the effectiveness of our method in a disparity post-processing step, we employ three post-processing approaches; cost modulation, ground control points-based propagation, and aggregated ground control points-based propagation. Experimental results demonstrate that our method outperforms state-of-the-art confidence estimation methods on various benchmarks.",
author = "Sunok Kim and Dongbo Min and Seungryong Kim and Kwanghoon Sohn",
year = "2019",
month = "3",
day = "1",
doi = "10.1109/TIP.2018.2878325",
language = "English",
volume = "28",
pages = "1299--1313",
journal = "IEEE Transactions on Image Processing",
issn = "1057-7149",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "3",

}

Unified confidence estimation networks for robust stereo matching. / Kim, Sunok; Min, Dongbo; Kim, Seungryong; Sohn, Kwanghoon.

In: IEEE Transactions on Image Processing, Vol. 28, No. 3, 8510870, 01.03.2019, p. 1299-1313.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Unified confidence estimation networks for robust stereo matching

AU - Kim, Sunok

AU - Min, Dongbo

AU - Kim, Seungryong

AU - Sohn, Kwanghoon

PY - 2019/3/1

Y1 - 2019/3/1

N2 - We present a deep architecture that estimates a stereo confidence, which is essential for improving the accuracy of stereo matching algorithms. In contrast to existing methods based on deep convolutional neural networks (CNNs) that rely on only one of the matching cost volume or estimated disparity map, our network estimates the stereo confidence by using the two heterogeneous inputs simultaneously. Specifically, the matching probability volume is first computed from the matching cost volume with residual networks and a pooling module in a manner that yields greater robustness. The confidence is then estimated through a unified deep network that combines confidence features extracted both from the matching probability volume and its corresponding disparity. In addition, our method extracts the confidence features of the disparity map by applying multiple convolutional filters with varying sizes to an input disparity map. To learn our networks in a semi-supervised manner, we propose a novel loss function that use confident points to compute the image reconstruction loss. To validate the effectiveness of our method in a disparity post-processing step, we employ three post-processing approaches; cost modulation, ground control points-based propagation, and aggregated ground control points-based propagation. Experimental results demonstrate that our method outperforms state-of-the-art confidence estimation methods on various benchmarks.

AB - We present a deep architecture that estimates a stereo confidence, which is essential for improving the accuracy of stereo matching algorithms. In contrast to existing methods based on deep convolutional neural networks (CNNs) that rely on only one of the matching cost volume or estimated disparity map, our network estimates the stereo confidence by using the two heterogeneous inputs simultaneously. Specifically, the matching probability volume is first computed from the matching cost volume with residual networks and a pooling module in a manner that yields greater robustness. The confidence is then estimated through a unified deep network that combines confidence features extracted both from the matching probability volume and its corresponding disparity. In addition, our method extracts the confidence features of the disparity map by applying multiple convolutional filters with varying sizes to an input disparity map. To learn our networks in a semi-supervised manner, we propose a novel loss function that use confident points to compute the image reconstruction loss. To validate the effectiveness of our method in a disparity post-processing step, we employ three post-processing approaches; cost modulation, ground control points-based propagation, and aggregated ground control points-based propagation. Experimental results demonstrate that our method outperforms state-of-the-art confidence estimation methods on various benchmarks.

UR - http://www.scopus.com/inward/record.url?scp=85055692060&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85055692060&partnerID=8YFLogxK

U2 - 10.1109/TIP.2018.2878325

DO - 10.1109/TIP.2018.2878325

M3 - Article

C2 - 30371368

AN - SCOPUS:85055692060

VL - 28

SP - 1299

EP - 1313

JO - IEEE Transactions on Image Processing

JF - IEEE Transactions on Image Processing

SN - 1057-7149

IS - 3

M1 - 8510870

ER -