Deep self-correlation descriptor for dense cross-modal correspondence

Seungryong Kim, Dongbo Min, Stephen Lin, Kwanghoon Sohn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

We present a novel descriptor, called deep self-correlation (DSC), designed for establishing dense correspondences between images taken under different imaging modalities, such as different spectral ranges or lighting conditions. Motivated by local self-similarity (LSS), we formulate a novel descriptor by leveraging LSS in a deep architecture, leading to better discriminative power and greater robustness to non-rigid image deformations than state-of-the-art descriptors. The DSC first computes self-correlation surfaces over a local support window for randomly sampled patches, and then builds hierarchical self-correlation surfaces by performing an average pooling within a deep architecture. Finally, the feature responses on the self-correlation surfaces are encoded through a spatial pyramid pooling in a circular configuration. In contrast to convolutional neural networks (CNNs) based descriptors, the DSC is trainingfree, is robust to cross-modal imaging, and can be densely computed in an efficient manner that significantly reduces computational redundancy. The state-of-the-art performance of DSC on challenging cases of cross-modal image pairs is demonstrated through extensive experiments.

Original languageEnglish
Title of host publicationComputer Vision - 14th European Conference, ECCV 2016, Proceedings
EditorsBastian Leibe, Jiri Matas, Max Welling, Nicu Sebe
PublisherSpringer Verlag
Pages679-695
Number of pages17
ISBN (Print)9783319464831
DOIs
Publication statusPublished - 2016 Jan 1

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9912 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Fingerprint

Descriptors
Correspondence
Imaging techniques
Pooling
Self-similarity
Redundancy
Lighting
Neural networks
Imaging
Pyramid
Modality
Patch
Experiments
Neural Networks
Robustness
Configuration
Range of data
Experiment

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Kim, S., Min, D., Lin, S., & Sohn, K. (2016). Deep self-correlation descriptor for dense cross-modal correspondence. In B. Leibe, J. Matas, M. Welling, & N. Sebe (Eds.), Computer Vision - 14th European Conference, ECCV 2016, Proceedings (pp. 679-695). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9912 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-46484-8_41
Kim, Seungryong ; Min, Dongbo ; Lin, Stephen ; Sohn, Kwanghoon. / Deep self-correlation descriptor for dense cross-modal correspondence. Computer Vision - 14th European Conference, ECCV 2016, Proceedings. editor / Bastian Leibe ; Jiri Matas ; Max Welling ; Nicu Sebe. Springer Verlag, 2016. pp. 679-695 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{169259d7eaf04f2a9c2d62b85125de8c,
title = "Deep self-correlation descriptor for dense cross-modal correspondence",
abstract = "We present a novel descriptor, called deep self-correlation (DSC), designed for establishing dense correspondences between images taken under different imaging modalities, such as different spectral ranges or lighting conditions. Motivated by local self-similarity (LSS), we formulate a novel descriptor by leveraging LSS in a deep architecture, leading to better discriminative power and greater robustness to non-rigid image deformations than state-of-the-art descriptors. The DSC first computes self-correlation surfaces over a local support window for randomly sampled patches, and then builds hierarchical self-correlation surfaces by performing an average pooling within a deep architecture. Finally, the feature responses on the self-correlation surfaces are encoded through a spatial pyramid pooling in a circular configuration. In contrast to convolutional neural networks (CNNs) based descriptors, the DSC is trainingfree, is robust to cross-modal imaging, and can be densely computed in an efficient manner that significantly reduces computational redundancy. The state-of-the-art performance of DSC on challenging cases of cross-modal image pairs is demonstrated through extensive experiments.",
author = "Seungryong Kim and Dongbo Min and Stephen Lin and Kwanghoon Sohn",
year = "2016",
month = "1",
day = "1",
doi = "10.1007/978-3-319-46484-8_41",
language = "English",
isbn = "9783319464831",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "679--695",
editor = "Bastian Leibe and Jiri Matas and Max Welling and Nicu Sebe",
booktitle = "Computer Vision - 14th European Conference, ECCV 2016, Proceedings",
address = "Germany",

}

Kim, S, Min, D, Lin, S & Sohn, K 2016, Deep self-correlation descriptor for dense cross-modal correspondence. in B Leibe, J Matas, M Welling & N Sebe (eds), Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9912 LNCS, Springer Verlag, pp. 679-695. https://doi.org/10.1007/978-3-319-46484-8_41

Deep self-correlation descriptor for dense cross-modal correspondence. / Kim, Seungryong; Min, Dongbo; Lin, Stephen; Sohn, Kwanghoon.

Computer Vision - 14th European Conference, ECCV 2016, Proceedings. ed. / Bastian Leibe; Jiri Matas; Max Welling; Nicu Sebe. Springer Verlag, 2016. p. 679-695 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9912 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Deep self-correlation descriptor for dense cross-modal correspondence

AU - Kim, Seungryong

AU - Min, Dongbo

AU - Lin, Stephen

AU - Sohn, Kwanghoon

PY - 2016/1/1

Y1 - 2016/1/1

N2 - We present a novel descriptor, called deep self-correlation (DSC), designed for establishing dense correspondences between images taken under different imaging modalities, such as different spectral ranges or lighting conditions. Motivated by local self-similarity (LSS), we formulate a novel descriptor by leveraging LSS in a deep architecture, leading to better discriminative power and greater robustness to non-rigid image deformations than state-of-the-art descriptors. The DSC first computes self-correlation surfaces over a local support window for randomly sampled patches, and then builds hierarchical self-correlation surfaces by performing an average pooling within a deep architecture. Finally, the feature responses on the self-correlation surfaces are encoded through a spatial pyramid pooling in a circular configuration. In contrast to convolutional neural networks (CNNs) based descriptors, the DSC is trainingfree, is robust to cross-modal imaging, and can be densely computed in an efficient manner that significantly reduces computational redundancy. The state-of-the-art performance of DSC on challenging cases of cross-modal image pairs is demonstrated through extensive experiments.

AB - We present a novel descriptor, called deep self-correlation (DSC), designed for establishing dense correspondences between images taken under different imaging modalities, such as different spectral ranges or lighting conditions. Motivated by local self-similarity (LSS), we formulate a novel descriptor by leveraging LSS in a deep architecture, leading to better discriminative power and greater robustness to non-rigid image deformations than state-of-the-art descriptors. The DSC first computes self-correlation surfaces over a local support window for randomly sampled patches, and then builds hierarchical self-correlation surfaces by performing an average pooling within a deep architecture. Finally, the feature responses on the self-correlation surfaces are encoded through a spatial pyramid pooling in a circular configuration. In contrast to convolutional neural networks (CNNs) based descriptors, the DSC is trainingfree, is robust to cross-modal imaging, and can be densely computed in an efficient manner that significantly reduces computational redundancy. The state-of-the-art performance of DSC on challenging cases of cross-modal image pairs is demonstrated through extensive experiments.

UR - http://www.scopus.com/inward/record.url?scp=84990028965&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84990028965&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-46484-8_41

DO - 10.1007/978-3-319-46484-8_41

M3 - Conference contribution

AN - SCOPUS:84990028965

SN - 9783319464831

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 679

EP - 695

BT - Computer Vision - 14th European Conference, ECCV 2016, Proceedings

A2 - Leibe, Bastian

A2 - Matas, Jiri

A2 - Welling, Max

A2 - Sebe, Nicu

PB - Springer Verlag

ER -

Kim S, Min D, Lin S, Sohn K. Deep self-correlation descriptor for dense cross-modal correspondence. In Leibe B, Matas J, Welling M, Sebe N, editors, Computer Vision - 14th European Conference, ECCV 2016, Proceedings. Springer Verlag. 2016. p. 679-695. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-46484-8_41