Proposal Flow: Semantic Correspondences from Object Proposals

Bumsub Ham, Minsu Cho, Cordelia Schmid, Jean Ponce

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

Finding image correspondences remains a challenging problem in the presence of intra-class variations and large changes in scene layout. Semantic flow methods are designed to handle images depicting different instances of the same object or scene category. We introduce a novel approach to semantic flow, dubbed proposal flow, that establishes reliable correspondences using object proposals. Unlike prevailing semantic flow approaches that operate on pixels or regularly sampled local regions, proposal flow benefits from the characteristics of modern object proposals, that exhibit high repeatability at multiple scales, and can take advantage of both local and geometric consistency constraints among proposals. We also show that the corresponding sparse proposal flow can effectively be transformed into a conventional dense flow field. We introduce two new challenging datasets that can be used to evaluate both general semantic flow techniques and region-based approaches such as proposal flow. We use these benchmarks to compare different matching algorithms, object proposals, and region features within proposal flow, to the state of the art in semantic flow. This comparison, along with experiments on standard datasets, demonstrates that proposal flow significantly outperforms existing semantic flow methods in various settings.

Original languageEnglish
Pages (from-to)1711-1725
Number of pages15
JournalIEEE transactions on pattern analysis and machine intelligence
Volume40
Issue number7
DOIs
Publication statusPublished - 2018 Jul 1

Fingerprint

Correspondence
Semantics
Object
Flow fields
Pixels
Multiple Scales
Repeatability
Matching Algorithm
Flow Field
Layout
Pixel
Experiments
Benchmark
Evaluate

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

Cite this

Ham, Bumsub ; Cho, Minsu ; Schmid, Cordelia ; Ponce, Jean. / Proposal Flow : Semantic Correspondences from Object Proposals. In: IEEE transactions on pattern analysis and machine intelligence. 2018 ; Vol. 40, No. 7. pp. 1711-1725.
@article{9d8512215d134e21b9e74c512e90a520,
title = "Proposal Flow: Semantic Correspondences from Object Proposals",
abstract = "Finding image correspondences remains a challenging problem in the presence of intra-class variations and large changes in scene layout. Semantic flow methods are designed to handle images depicting different instances of the same object or scene category. We introduce a novel approach to semantic flow, dubbed proposal flow, that establishes reliable correspondences using object proposals. Unlike prevailing semantic flow approaches that operate on pixels or regularly sampled local regions, proposal flow benefits from the characteristics of modern object proposals, that exhibit high repeatability at multiple scales, and can take advantage of both local and geometric consistency constraints among proposals. We also show that the corresponding sparse proposal flow can effectively be transformed into a conventional dense flow field. We introduce two new challenging datasets that can be used to evaluate both general semantic flow techniques and region-based approaches such as proposal flow. We use these benchmarks to compare different matching algorithms, object proposals, and region features within proposal flow, to the state of the art in semantic flow. This comparison, along with experiments on standard datasets, demonstrates that proposal flow significantly outperforms existing semantic flow methods in various settings.",
author = "Bumsub Ham and Minsu Cho and Cordelia Schmid and Jean Ponce",
year = "2018",
month = "7",
day = "1",
doi = "10.1109/TPAMI.2017.2724510",
language = "English",
volume = "40",
pages = "1711--1725",
journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",
issn = "0162-8828",
publisher = "IEEE Computer Society",
number = "7",

}

Proposal Flow : Semantic Correspondences from Object Proposals. / Ham, Bumsub; Cho, Minsu; Schmid, Cordelia; Ponce, Jean.

In: IEEE transactions on pattern analysis and machine intelligence, Vol. 40, No. 7, 01.07.2018, p. 1711-1725.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Proposal Flow

T2 - Semantic Correspondences from Object Proposals

AU - Ham, Bumsub

AU - Cho, Minsu

AU - Schmid, Cordelia

AU - Ponce, Jean

PY - 2018/7/1

Y1 - 2018/7/1

N2 - Finding image correspondences remains a challenging problem in the presence of intra-class variations and large changes in scene layout. Semantic flow methods are designed to handle images depicting different instances of the same object or scene category. We introduce a novel approach to semantic flow, dubbed proposal flow, that establishes reliable correspondences using object proposals. Unlike prevailing semantic flow approaches that operate on pixels or regularly sampled local regions, proposal flow benefits from the characteristics of modern object proposals, that exhibit high repeatability at multiple scales, and can take advantage of both local and geometric consistency constraints among proposals. We also show that the corresponding sparse proposal flow can effectively be transformed into a conventional dense flow field. We introduce two new challenging datasets that can be used to evaluate both general semantic flow techniques and region-based approaches such as proposal flow. We use these benchmarks to compare different matching algorithms, object proposals, and region features within proposal flow, to the state of the art in semantic flow. This comparison, along with experiments on standard datasets, demonstrates that proposal flow significantly outperforms existing semantic flow methods in various settings.

AB - Finding image correspondences remains a challenging problem in the presence of intra-class variations and large changes in scene layout. Semantic flow methods are designed to handle images depicting different instances of the same object or scene category. We introduce a novel approach to semantic flow, dubbed proposal flow, that establishes reliable correspondences using object proposals. Unlike prevailing semantic flow approaches that operate on pixels or regularly sampled local regions, proposal flow benefits from the characteristics of modern object proposals, that exhibit high repeatability at multiple scales, and can take advantage of both local and geometric consistency constraints among proposals. We also show that the corresponding sparse proposal flow can effectively be transformed into a conventional dense flow field. We introduce two new challenging datasets that can be used to evaluate both general semantic flow techniques and region-based approaches such as proposal flow. We use these benchmarks to compare different matching algorithms, object proposals, and region features within proposal flow, to the state of the art in semantic flow. This comparison, along with experiments on standard datasets, demonstrates that proposal flow significantly outperforms existing semantic flow methods in various settings.

UR - http://www.scopus.com/inward/record.url?scp=85023646116&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85023646116&partnerID=8YFLogxK

U2 - 10.1109/TPAMI.2017.2724510

DO - 10.1109/TPAMI.2017.2724510

M3 - Article

C2 - 28708543

AN - SCOPUS:85023646116

VL - 40

SP - 1711

EP - 1725

JO - IEEE Transactions on Pattern Analysis and Machine Intelligence

JF - IEEE Transactions on Pattern Analysis and Machine Intelligence

SN - 0162-8828

IS - 7

ER -