Feature Augmentation for Learning Confidence Measure in Stereo Matching

Sunok Kim, Dongbo Min, Seungryong Kim, Kwanghoon Sohn

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Confidence estimation is essential for refining stereo matching results through a post-processing step. This problem has recently been studied using a learning-based approach, which demonstrates a substantial improvement on conventional simple non-learning based methods. However, the formulation of learning-based methods that individually estimates the confidence of each pixel disregards spatial coherency that might exist in the confidence map, thus providing a limited performance under challenging conditions. Our key observation is that the confidence features and resulting confidence maps are smoothly varying in the spatial domain, and highly correlated within the local regions of an image. We present a new approach that imposes spatial consistency on the confidence estimation. Specifically, a set of robust confidence features is extracted from each superpixel decomposed using the Gaussian mixture model, and then these features are concatenated with pixel-level confidence features. The features are then enhanced through adaptive filtering in the feature domain. In addition, the resulting confidence map, estimated using the confidence features with a random regression forest, is further improved through K-nearest neighbor based aggregation scheme on both pixel- and superpixel-level. To validate the proposed confidence estimation scheme, we employ cost modulation or ground control points based optimization in stereo matching. Experimental results demonstrate that the proposed method outperforms state-of-the-art approaches on various benchmarks including challenging outdoor scenes.

Original languageEnglish
Article number8030080
Pages (from-to)6019-6033
Number of pages15
JournalIEEE Transactions on Image Processing
Volume26
Issue number12
DOIs
Publication statusPublished - 2017 Dec

Fingerprint

Pixels
Adaptive filtering
Refining
Agglomeration
Modulation
Processing
Costs

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Graphics and Computer-Aided Design

Cite this

Kim, Sunok ; Min, Dongbo ; Kim, Seungryong ; Sohn, Kwanghoon. / Feature Augmentation for Learning Confidence Measure in Stereo Matching. In: IEEE Transactions on Image Processing. 2017 ; Vol. 26, No. 12. pp. 6019-6033.
@article{4c134e9594a543b791195655811f92ba,
title = "Feature Augmentation for Learning Confidence Measure in Stereo Matching",
abstract = "Confidence estimation is essential for refining stereo matching results through a post-processing step. This problem has recently been studied using a learning-based approach, which demonstrates a substantial improvement on conventional simple non-learning based methods. However, the formulation of learning-based methods that individually estimates the confidence of each pixel disregards spatial coherency that might exist in the confidence map, thus providing a limited performance under challenging conditions. Our key observation is that the confidence features and resulting confidence maps are smoothly varying in the spatial domain, and highly correlated within the local regions of an image. We present a new approach that imposes spatial consistency on the confidence estimation. Specifically, a set of robust confidence features is extracted from each superpixel decomposed using the Gaussian mixture model, and then these features are concatenated with pixel-level confidence features. The features are then enhanced through adaptive filtering in the feature domain. In addition, the resulting confidence map, estimated using the confidence features with a random regression forest, is further improved through K-nearest neighbor based aggregation scheme on both pixel- and superpixel-level. To validate the proposed confidence estimation scheme, we employ cost modulation or ground control points based optimization in stereo matching. Experimental results demonstrate that the proposed method outperforms state-of-the-art approaches on various benchmarks including challenging outdoor scenes.",
author = "Sunok Kim and Dongbo Min and Seungryong Kim and Kwanghoon Sohn",
year = "2017",
month = "12",
doi = "10.1109/TIP.2017.2750404",
language = "English",
volume = "26",
pages = "6019--6033",
journal = "IEEE Transactions on Image Processing",
issn = "1057-7149",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "12",

}

Feature Augmentation for Learning Confidence Measure in Stereo Matching. / Kim, Sunok; Min, Dongbo; Kim, Seungryong; Sohn, Kwanghoon.

In: IEEE Transactions on Image Processing, Vol. 26, No. 12, 8030080, 12.2017, p. 6019-6033.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Feature Augmentation for Learning Confidence Measure in Stereo Matching

AU - Kim, Sunok

AU - Min, Dongbo

AU - Kim, Seungryong

AU - Sohn, Kwanghoon

PY - 2017/12

Y1 - 2017/12

N2 - Confidence estimation is essential for refining stereo matching results through a post-processing step. This problem has recently been studied using a learning-based approach, which demonstrates a substantial improvement on conventional simple non-learning based methods. However, the formulation of learning-based methods that individually estimates the confidence of each pixel disregards spatial coherency that might exist in the confidence map, thus providing a limited performance under challenging conditions. Our key observation is that the confidence features and resulting confidence maps are smoothly varying in the spatial domain, and highly correlated within the local regions of an image. We present a new approach that imposes spatial consistency on the confidence estimation. Specifically, a set of robust confidence features is extracted from each superpixel decomposed using the Gaussian mixture model, and then these features are concatenated with pixel-level confidence features. The features are then enhanced through adaptive filtering in the feature domain. In addition, the resulting confidence map, estimated using the confidence features with a random regression forest, is further improved through K-nearest neighbor based aggregation scheme on both pixel- and superpixel-level. To validate the proposed confidence estimation scheme, we employ cost modulation or ground control points based optimization in stereo matching. Experimental results demonstrate that the proposed method outperforms state-of-the-art approaches on various benchmarks including challenging outdoor scenes.

AB - Confidence estimation is essential for refining stereo matching results through a post-processing step. This problem has recently been studied using a learning-based approach, which demonstrates a substantial improvement on conventional simple non-learning based methods. However, the formulation of learning-based methods that individually estimates the confidence of each pixel disregards spatial coherency that might exist in the confidence map, thus providing a limited performance under challenging conditions. Our key observation is that the confidence features and resulting confidence maps are smoothly varying in the spatial domain, and highly correlated within the local regions of an image. We present a new approach that imposes spatial consistency on the confidence estimation. Specifically, a set of robust confidence features is extracted from each superpixel decomposed using the Gaussian mixture model, and then these features are concatenated with pixel-level confidence features. The features are then enhanced through adaptive filtering in the feature domain. In addition, the resulting confidence map, estimated using the confidence features with a random regression forest, is further improved through K-nearest neighbor based aggregation scheme on both pixel- and superpixel-level. To validate the proposed confidence estimation scheme, we employ cost modulation or ground control points based optimization in stereo matching. Experimental results demonstrate that the proposed method outperforms state-of-the-art approaches on various benchmarks including challenging outdoor scenes.

UR - http://www.scopus.com/inward/record.url?scp=85032381267&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85032381267&partnerID=8YFLogxK

U2 - 10.1109/TIP.2017.2750404

DO - 10.1109/TIP.2017.2750404

M3 - Article

C2 - 28910763

AN - SCOPUS:85032381267

VL - 26

SP - 6019

EP - 6033

JO - IEEE Transactions on Image Processing

JF - IEEE Transactions on Image Processing

SN - 1057-7149

IS - 12

M1 - 8030080

ER -