PARN

Pyramidal affine regression networks for dense semantic correspondence

Sangryul Jeon, Seungryong Kim, Dongbo Min, Kwanghoon Sohn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper presents a deep architecture for dense semantic correspondence, called pyramidal affine regression networks (PARN), that estimates locally-varying affine transformation fields across images. To deal with intra-class appearance and shape variations that commonly exist among different instances within the same object category, we leverage a pyramidal model where affine transformation fields are progressively estimated in a coarse-to-fine manner so that the smoothness constraint is naturally imposed within deep networks. PARN estimates residual affine transformations at each level and composes them to estimate final affine transformations. Furthermore, to overcome the limitations of insufficient training data for semantic correspondence, we propose a novel weakly-supervised training scheme that generates progressive supervisions by leveraging a correspondence consistency across image pairs. Our method is fully learnable in an end-to-end manner and does not require quantizing infinite continuous affine transformation fields. To the best of our knowledge, it is the first work that attempts to estimate dense affine transformation fields in a coarse-to-fine manner within deep networks. Experimental results demonstrate that PARN outperforms the state-of-the-art methods for dense semantic correspondence on various benchmarks.

Original languageEnglish
Title of host publicationComputer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings
EditorsMartial Hebert, Yair Weiss, Vittorio Ferrari, Cristian Sminchisescu
PublisherSpringer Verlag
Pages355-371
Number of pages17
ISBN (Print)9783030012304
DOIs
Publication statusPublished - 2018 Jan 1
Event15th European Conference on Computer Vision, ECCV 2018 - Munich, Germany
Duration: 2018 Sep 82018 Sep 14

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11210 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other15th European Conference on Computer Vision, ECCV 2018
CountryGermany
CityMunich
Period18/9/818/9/14

Fingerprint

Affine transformation
Correspondence
Regression
Semantics
Estimate
Leverage
Smoothness
Benchmark
Experimental Results
Demonstrate
Training

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Jeon, S., Kim, S., Min, D., & Sohn, K. (2018). PARN: Pyramidal affine regression networks for dense semantic correspondence. In M. Hebert, Y. Weiss, V. Ferrari, & C. Sminchisescu (Eds.), Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings (pp. 355-371). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11210 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-01231-1_22
Jeon, Sangryul ; Kim, Seungryong ; Min, Dongbo ; Sohn, Kwanghoon. / PARN : Pyramidal affine regression networks for dense semantic correspondence. Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. editor / Martial Hebert ; Yair Weiss ; Vittorio Ferrari ; Cristian Sminchisescu. Springer Verlag, 2018. pp. 355-371 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{ad14b7f0b5bc474195178c0e7fcf074f,
title = "PARN: Pyramidal affine regression networks for dense semantic correspondence",
abstract = "This paper presents a deep architecture for dense semantic correspondence, called pyramidal affine regression networks (PARN), that estimates locally-varying affine transformation fields across images. To deal with intra-class appearance and shape variations that commonly exist among different instances within the same object category, we leverage a pyramidal model where affine transformation fields are progressively estimated in a coarse-to-fine manner so that the smoothness constraint is naturally imposed within deep networks. PARN estimates residual affine transformations at each level and composes them to estimate final affine transformations. Furthermore, to overcome the limitations of insufficient training data for semantic correspondence, we propose a novel weakly-supervised training scheme that generates progressive supervisions by leveraging a correspondence consistency across image pairs. Our method is fully learnable in an end-to-end manner and does not require quantizing infinite continuous affine transformation fields. To the best of our knowledge, it is the first work that attempts to estimate dense affine transformation fields in a coarse-to-fine manner within deep networks. Experimental results demonstrate that PARN outperforms the state-of-the-art methods for dense semantic correspondence on various benchmarks.",
author = "Sangryul Jeon and Seungryong Kim and Dongbo Min and Kwanghoon Sohn",
year = "2018",
month = "1",
day = "1",
doi = "10.1007/978-3-030-01231-1_22",
language = "English",
isbn = "9783030012304",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "355--371",
editor = "Martial Hebert and Yair Weiss and Vittorio Ferrari and Cristian Sminchisescu",
booktitle = "Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings",
address = "Germany",

}

Jeon, S, Kim, S, Min, D & Sohn, K 2018, PARN: Pyramidal affine regression networks for dense semantic correspondence. in M Hebert, Y Weiss, V Ferrari & C Sminchisescu (eds), Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11210 LNCS, Springer Verlag, pp. 355-371, 15th European Conference on Computer Vision, ECCV 2018, Munich, Germany, 18/9/8. https://doi.org/10.1007/978-3-030-01231-1_22

PARN : Pyramidal affine regression networks for dense semantic correspondence. / Jeon, Sangryul; Kim, Seungryong; Min, Dongbo; Sohn, Kwanghoon.

Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. ed. / Martial Hebert; Yair Weiss; Vittorio Ferrari; Cristian Sminchisescu. Springer Verlag, 2018. p. 355-371 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11210 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - PARN

T2 - Pyramidal affine regression networks for dense semantic correspondence

AU - Jeon, Sangryul

AU - Kim, Seungryong

AU - Min, Dongbo

AU - Sohn, Kwanghoon

PY - 2018/1/1

Y1 - 2018/1/1

N2 - This paper presents a deep architecture for dense semantic correspondence, called pyramidal affine regression networks (PARN), that estimates locally-varying affine transformation fields across images. To deal with intra-class appearance and shape variations that commonly exist among different instances within the same object category, we leverage a pyramidal model where affine transformation fields are progressively estimated in a coarse-to-fine manner so that the smoothness constraint is naturally imposed within deep networks. PARN estimates residual affine transformations at each level and composes them to estimate final affine transformations. Furthermore, to overcome the limitations of insufficient training data for semantic correspondence, we propose a novel weakly-supervised training scheme that generates progressive supervisions by leveraging a correspondence consistency across image pairs. Our method is fully learnable in an end-to-end manner and does not require quantizing infinite continuous affine transformation fields. To the best of our knowledge, it is the first work that attempts to estimate dense affine transformation fields in a coarse-to-fine manner within deep networks. Experimental results demonstrate that PARN outperforms the state-of-the-art methods for dense semantic correspondence on various benchmarks.

AB - This paper presents a deep architecture for dense semantic correspondence, called pyramidal affine regression networks (PARN), that estimates locally-varying affine transformation fields across images. To deal with intra-class appearance and shape variations that commonly exist among different instances within the same object category, we leverage a pyramidal model where affine transformation fields are progressively estimated in a coarse-to-fine manner so that the smoothness constraint is naturally imposed within deep networks. PARN estimates residual affine transformations at each level and composes them to estimate final affine transformations. Furthermore, to overcome the limitations of insufficient training data for semantic correspondence, we propose a novel weakly-supervised training scheme that generates progressive supervisions by leveraging a correspondence consistency across image pairs. Our method is fully learnable in an end-to-end manner and does not require quantizing infinite continuous affine transformation fields. To the best of our knowledge, it is the first work that attempts to estimate dense affine transformation fields in a coarse-to-fine manner within deep networks. Experimental results demonstrate that PARN outperforms the state-of-the-art methods for dense semantic correspondence on various benchmarks.

UR - http://www.scopus.com/inward/record.url?scp=85055125689&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85055125689&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-01231-1_22

DO - 10.1007/978-3-030-01231-1_22

M3 - Conference contribution

SN - 9783030012304

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 355

EP - 371

BT - Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings

A2 - Hebert, Martial

A2 - Weiss, Yair

A2 - Ferrari, Vittorio

A2 - Sminchisescu, Cristian

PB - Springer Verlag

ER -

Jeon S, Kim S, Min D, Sohn K. PARN: Pyramidal affine regression networks for dense semantic correspondence. In Hebert M, Weiss Y, Ferrari V, Sminchisescu C, editors, Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. Springer Verlag. 2018. p. 355-371. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-01231-1_22