Deep Semantic Matching with Foreground Detection and Cycle-Consistency

Yun Chun Chen, Po Hsiang Huang, Li Yu Yu, Jia Bin Huang, Ming Hsuan Yang, Yen Yu Lin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Establishing dense semantic correspondences between object instances remains a challenging problem due to background clutter, significant scale and pose differences, and large intra-class variations. In this paper, we present an end-to-end trainable network for learning semantic correspondences using only matching image pairs without manual keypoint correspondence annotations. To facilitate network training with this weaker form of supervision, we (1) explicitly estimate the foreground regions to suppress the effect of background clutter and (2) develop cycle-consistent losses to enforce the predicted transformations across multiple images to be geometrically plausible and consistent. We train the proposed model using the PF-PASCAL dataset and evaluate the performance on the PF-PASCAL, PF-WILLOW, and TSS datasets. Extensive experimental results show that the proposed approach achieves favorably performance compared to the state-of-the-art. The code and model will be available at https://yunchunchen.github.io/WeakMatchNet/.

Original languageEnglish
Title of host publicationComputer Vision - ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers
EditorsC.V. Jawahar, Konrad Schindler, Hongdong Li, Greg Mori
PublisherSpringer Verlag
Pages347-362
Number of pages16
ISBN (Print)9783030208929
DOIs
Publication statusPublished - 2019
Event14th Asian Conference on Computer Vision, ACCV 2018 - Perth, Australia
Duration: 2018 Dec 22018 Dec 6

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11363 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference14th Asian Conference on Computer Vision, ACCV 2018
CountryAustralia
CityPerth
Period18/12/218/12/6

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Deep Semantic Matching with Foreground Detection and Cycle-Consistency'. Together they form a unique fingerprint.

  • Cite this

    Chen, Y. C., Huang, P. H., Yu, L. Y., Huang, J. B., Yang, M. H., & Lin, Y. Y. (2019). Deep Semantic Matching with Foreground Detection and Cycle-Consistency. In C. V. Jawahar, K. Schindler, H. Li, & G. Mori (Eds.), Computer Vision - ACCV 2018 - 14th Asian Conference on Computer Vision, Revised Selected Papers (pp. 347-362). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11363 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-20893-6_22