DFT-based transformation invariant pooling layer for visual classification

Jongbin Ryu; Ming Hsuan Yang; Jongwoo Lim

doi:10.1007/978-3-030-01264-9_6

DFT-based transformation invariant pooling layer for visual classification

Jongbin Ryu, Ming Hsuan Yang, Jongwoo Lim

Department of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

4 Citations (Scopus)

Abstract

We propose a novel discrete Fourier transform-based pooling layer for convolutional neural networks. The DFT magnitude pooling replaces the traditional max/average pooling layer between the convolution and fully-connected layers to retain translation invariance and shape preserving (aware of shape difference) properties based on the shift theorem of the Fourier transform. Thanks to the ability to handle image misalignment while keeping important structural information in the pooling stage, the DFT magnitude pooling improves the classification accuracy significantly. In addition, we propose the DFT⁺ method for ensemble networks using the middle convolution layer outputs. The proposed methods are extensively evaluated on various classification tasks using the ImageNet, CUB 2010-2011, MIT Indoors, Caltech 101, FMD and DTD datasets. The AlexNet, VGG-VD 16, Inception-v3, and ResNet are used as the base networks, upon which DFT and DFT⁺ methods are implemented. Experimental results show that the proposed methods improve the classification performance in all networks and datasets.

Original language	English
Title of host publication	Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings
Editors	Vittorio Ferrari, Cristian Sminchisescu, Yair Weiss, Martial Hebert
Publisher	Springer Verlag
Pages	89-104
Number of pages	16
ISBN (Print)	9783030012632
DOIs	https://doi.org/10.1007/978-3-030-01264-9_6
Publication status	Published - 2018
Event	15th European Conference on Computer Vision, ECCV 2018 - Munich, Germany Duration: 2018 Sep 8 → 2018 Sep 14

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11218 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	15th European Conference on Computer Vision, ECCV 2018
Country/Territory	Germany
City	Munich
Period	18/9/8 → 18/9/14

Bibliographical note

Funding Information:
Acknowledgements. This work was partially supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education (NRF-2017R1A6A3A11031193), Next-Generation Information Computing Development Program through the NRF funded by the Ministry of Science, ICT (NRF-2017M3C4A7069366) and the NSF CAREER Grant #1149783.

Publisher Copyright:
© 2018, Springer Nature Switzerland AG.

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-030-01264-9_6

Cite this

Ryu, J., Yang, M. H., & Lim, J. (2018). DFT-based transformation invariant pooling layer for visual classification. In V. Ferrari, C. Sminchisescu, Y. Weiss, & M. Hebert (Eds.), Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings (pp. 89-104). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11218 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-01264-9_6

Ryu, Jongbin ; Yang, Ming Hsuan ; Lim, Jongwoo. / DFT-based transformation invariant pooling layer for visual classification. Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. editor / Vittorio Ferrari ; Cristian Sminchisescu ; Yair Weiss ; Martial Hebert. Springer Verlag, 2018. pp. 89-104 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{37b81fc775a84c49b97a98bee4225902,

title = "DFT-based transformation invariant pooling layer for visual classification",

abstract = "We propose a novel discrete Fourier transform-based pooling layer for convolutional neural networks. The DFT magnitude pooling replaces the traditional max/average pooling layer between the convolution and fully-connected layers to retain translation invariance and shape preserving (aware of shape difference) properties based on the shift theorem of the Fourier transform. Thanks to the ability to handle image misalignment while keeping important structural information in the pooling stage, the DFT magnitude pooling improves the classification accuracy significantly. In addition, we propose the DFT+ method for ensemble networks using the middle convolution layer outputs. The proposed methods are extensively evaluated on various classification tasks using the ImageNet, CUB 2010-2011, MIT Indoors, Caltech 101, FMD and DTD datasets. The AlexNet, VGG-VD 16, Inception-v3, and ResNet are used as the base networks, upon which DFT and DFT+ methods are implemented. Experimental results show that the proposed methods improve the classification performance in all networks and datasets.",

author = "Jongbin Ryu and Yang, {Ming Hsuan} and Jongwoo Lim",

note = "Funding Information: Acknowledgements. This work was partially supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education (NRF-2017R1A6A3A11031193), Next-Generation Information Computing Development Program through the NRF funded by the Ministry of Science, ICT (NRF-2017M3C4A7069366) and the NSF CAREER Grant #1149783. Publisher Copyright: {\textcopyright} 2018, Springer Nature Switzerland AG.; 15th European Conference on Computer Vision, ECCV 2018 ; Conference date: 08-09-2018 Through 14-09-2018",

year = "2018",

doi = "10.1007/978-3-030-01264-9_6",

language = "English",

isbn = "9783030012632",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "89--104",

editor = "Vittorio Ferrari and Cristian Sminchisescu and Yair Weiss and Martial Hebert",

booktitle = "Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings",

address = "Germany",

}

Ryu, J, Yang, MH & Lim, J 2018, DFT-based transformation invariant pooling layer for visual classification. in V Ferrari, C Sminchisescu, Y Weiss & M Hebert (eds), Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11218 LNCS, Springer Verlag, pp. 89-104, 15th European Conference on Computer Vision, ECCV 2018, Munich, Germany, 18/9/8. https://doi.org/10.1007/978-3-030-01264-9_6

DFT-based transformation invariant pooling layer for visual classification. / Ryu, Jongbin; Yang, Ming Hsuan; Lim, Jongwoo.

Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. ed. / Vittorio Ferrari; Cristian Sminchisescu; Yair Weiss; Martial Hebert. Springer Verlag, 2018. p. 89-104 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11218 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - DFT-based transformation invariant pooling layer for visual classification

AU - Ryu, Jongbin

AU - Yang, Ming Hsuan

AU - Lim, Jongwoo

N1 - Funding Information: Acknowledgements. This work was partially supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education (NRF-2017R1A6A3A11031193), Next-Generation Information Computing Development Program through the NRF funded by the Ministry of Science, ICT (NRF-2017M3C4A7069366) and the NSF CAREER Grant #1149783. Publisher Copyright: © 2018, Springer Nature Switzerland AG.

PY - 2018

Y1 - 2018

N2 - We propose a novel discrete Fourier transform-based pooling layer for convolutional neural networks. The DFT magnitude pooling replaces the traditional max/average pooling layer between the convolution and fully-connected layers to retain translation invariance and shape preserving (aware of shape difference) properties based on the shift theorem of the Fourier transform. Thanks to the ability to handle image misalignment while keeping important structural information in the pooling stage, the DFT magnitude pooling improves the classification accuracy significantly. In addition, we propose the DFT+ method for ensemble networks using the middle convolution layer outputs. The proposed methods are extensively evaluated on various classification tasks using the ImageNet, CUB 2010-2011, MIT Indoors, Caltech 101, FMD and DTD datasets. The AlexNet, VGG-VD 16, Inception-v3, and ResNet are used as the base networks, upon which DFT and DFT+ methods are implemented. Experimental results show that the proposed methods improve the classification performance in all networks and datasets.

AB - We propose a novel discrete Fourier transform-based pooling layer for convolutional neural networks. The DFT magnitude pooling replaces the traditional max/average pooling layer between the convolution and fully-connected layers to retain translation invariance and shape preserving (aware of shape difference) properties based on the shift theorem of the Fourier transform. Thanks to the ability to handle image misalignment while keeping important structural information in the pooling stage, the DFT magnitude pooling improves the classification accuracy significantly. In addition, we propose the DFT+ method for ensemble networks using the middle convolution layer outputs. The proposed methods are extensively evaluated on various classification tasks using the ImageNet, CUB 2010-2011, MIT Indoors, Caltech 101, FMD and DTD datasets. The AlexNet, VGG-VD 16, Inception-v3, and ResNet are used as the base networks, upon which DFT and DFT+ methods are implemented. Experimental results show that the proposed methods improve the classification performance in all networks and datasets.

UR - http://www.scopus.com/inward/record.url?scp=85055696119&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85055696119&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-01264-9_6

DO - 10.1007/978-3-030-01264-9_6

M3 - Conference contribution

AN - SCOPUS:85055696119

SN - 9783030012632

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 89

EP - 104

BT - Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings

A2 - Ferrari, Vittorio

A2 - Sminchisescu, Cristian

A2 - Weiss, Yair

A2 - Hebert, Martial

PB - Springer Verlag

T2 - 15th European Conference on Computer Vision, ECCV 2018

Y2 - 8 September 2018 through 14 September 2018

ER -

Ryu J, Yang MH, Lim J. DFT-based transformation invariant pooling layer for visual classification. In Ferrari V, Sminchisescu C, Weiss Y, Hebert M, editors, Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. Springer Verlag. 2018. p. 89-104. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-01264-9_6

DFT-based transformation invariant pooling layer for visual classification

Abstract

Publication series

Other

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this