Discrete-Continuous Transformation Matching for Dense Semantic Correspondence

Seungryong Kim, Dongbo Min, Stephen Lin, Kwanghoon Sohn

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Techniques for dense semantic correspondence have provided limited ability to deal with the geometric variations that commonly exist between semantically similar images. While variations due to scale and rotation have been examined, there is a lack of practical solutions for more complex deformations such as affine transformations because of the tremendous size of the associated solution space. To address this problem, we present a discrete-continuous transformation matching (DCTM) framework where dense affine transformation fields are inferred through a discrete label optimization in which the labels are iteratively updated via continuous regularization. In this way, our approach draws solutions from the continuous space of affine transformations in a manner that can be computed efficiently through constant-time edge-aware filtering and a proposed affine-varying CNN-based descriptor. Furthermore, leveraging correspondence consistency and confidence-guided filtering in each iteration facilitates the convergence of our method. Experimental results show that this model outperforms the state-of-the-art methods for dense semantic correspondence on various benchmarks and applications.

Original languageEnglish
JournalIEEE transactions on pattern analysis and machine intelligence
DOIs
Publication statusAccepted/In press - 2018 Jan 1

Fingerprint

Affine transformation
Correspondence
Semantics
Labels
Filtering
Time Constant
Descriptors
Confidence
Regularization
Benchmark
Iteration
Optimization
Experimental Results
Model
Framework

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

Cite this

@article{4b5d0f4f130f4e2f9582d4a6b2a57355,
title = "Discrete-Continuous Transformation Matching for Dense Semantic Correspondence",
abstract = "Techniques for dense semantic correspondence have provided limited ability to deal with the geometric variations that commonly exist between semantically similar images. While variations due to scale and rotation have been examined, there is a lack of practical solutions for more complex deformations such as affine transformations because of the tremendous size of the associated solution space. To address this problem, we present a discrete-continuous transformation matching (DCTM) framework where dense affine transformation fields are inferred through a discrete label optimization in which the labels are iteratively updated via continuous regularization. In this way, our approach draws solutions from the continuous space of affine transformations in a manner that can be computed efficiently through constant-time edge-aware filtering and a proposed affine-varying CNN-based descriptor. Furthermore, leveraging correspondence consistency and confidence-guided filtering in each iteration facilitates the convergence of our method. Experimental results show that this model outperforms the state-of-the-art methods for dense semantic correspondence on various benchmarks and applications.",
author = "Seungryong Kim and Dongbo Min and Stephen Lin and Kwanghoon Sohn",
year = "2018",
month = "1",
day = "1",
doi = "10.1109/TPAMI.2018.2878240",
language = "English",
journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",
issn = "0162-8828",
publisher = "IEEE Computer Society",

}

Discrete-Continuous Transformation Matching for Dense Semantic Correspondence. / Kim, Seungryong; Min, Dongbo; Lin, Stephen; Sohn, Kwanghoon.

In: IEEE transactions on pattern analysis and machine intelligence, 01.01.2018.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Discrete-Continuous Transformation Matching for Dense Semantic Correspondence

AU - Kim, Seungryong

AU - Min, Dongbo

AU - Lin, Stephen

AU - Sohn, Kwanghoon

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Techniques for dense semantic correspondence have provided limited ability to deal with the geometric variations that commonly exist between semantically similar images. While variations due to scale and rotation have been examined, there is a lack of practical solutions for more complex deformations such as affine transformations because of the tremendous size of the associated solution space. To address this problem, we present a discrete-continuous transformation matching (DCTM) framework where dense affine transformation fields are inferred through a discrete label optimization in which the labels are iteratively updated via continuous regularization. In this way, our approach draws solutions from the continuous space of affine transformations in a manner that can be computed efficiently through constant-time edge-aware filtering and a proposed affine-varying CNN-based descriptor. Furthermore, leveraging correspondence consistency and confidence-guided filtering in each iteration facilitates the convergence of our method. Experimental results show that this model outperforms the state-of-the-art methods for dense semantic correspondence on various benchmarks and applications.

AB - Techniques for dense semantic correspondence have provided limited ability to deal with the geometric variations that commonly exist between semantically similar images. While variations due to scale and rotation have been examined, there is a lack of practical solutions for more complex deformations such as affine transformations because of the tremendous size of the associated solution space. To address this problem, we present a discrete-continuous transformation matching (DCTM) framework where dense affine transformation fields are inferred through a discrete label optimization in which the labels are iteratively updated via continuous regularization. In this way, our approach draws solutions from the continuous space of affine transformations in a manner that can be computed efficiently through constant-time edge-aware filtering and a proposed affine-varying CNN-based descriptor. Furthermore, leveraging correspondence consistency and confidence-guided filtering in each iteration facilitates the convergence of our method. Experimental results show that this model outperforms the state-of-the-art methods for dense semantic correspondence on various benchmarks and applications.

UR - http://www.scopus.com/inward/record.url?scp=85055701362&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85055701362&partnerID=8YFLogxK

U2 - 10.1109/TPAMI.2018.2878240

DO - 10.1109/TPAMI.2018.2878240

M3 - Article

JO - IEEE Transactions on Pattern Analysis and Machine Intelligence

JF - IEEE Transactions on Pattern Analysis and Machine Intelligence

SN - 0162-8828

ER -