Scene Recognition via Object-to-Scene Class Conversion: End-to-End Training

Hongje Seong, Junhyuk Hyun, Hyunbae Chang, Suhyeon Lee, Suhan Woo, Euntai Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

When a person recognize the scene of an image, contextual understanding from its environmental elements is necessary. These environmental elements are variant and require comprehensive understanding of various situations. Especially, objects are frequently used as environmental elements related with scene. In this paper, we suggest a score level Class Conversion Matrix (CCM) for scene recognition with a great focus on relationship between objects and scene. A lot of existing methods have already build scene recognition systems with consideration of close relationship between object and scenes. However, most of these methods are using the object features directly without any conversions or reconstructions, and it lack confirmation whether these object features are helpful to recognize scenes correctly. To solve this problem, CCM, a matrix converting object feature to scene feature, is suggested. Moreover, CCM can be implemented with neural network layer and end-to-end trainable. Extensive experiments on Places 2 dataset demonstrate the effectiveness of our approach, when it is applied to the existing deep convolutional neural network architectures. The code is available at https://github.com/Hongje/Class_Conversion_Matrix-Places365.

Original languageEnglish
Title of host publication2019 International Joint Conference on Neural Networks, IJCNN 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728119854
DOIs
Publication statusPublished - 2019 Jul
Event2019 International Joint Conference on Neural Networks, IJCNN 2019 - Budapest, Hungary
Duration: 2019 Jul 142019 Jul 19

Publication series

NameProceedings of the International Joint Conference on Neural Networks
Volume2019-July

Conference

Conference2019 International Joint Conference on Neural Networks, IJCNN 2019
CountryHungary
CityBudapest
Period19/7/1419/7/19

Fingerprint

Neural networks
Image understanding
Network layers
Network architecture
Experiments

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence

Cite this

Seong, H., Hyun, J., Chang, H., Lee, S., Woo, S., & Kim, E. (2019). Scene Recognition via Object-to-Scene Class Conversion: End-to-End Training. In 2019 International Joint Conference on Neural Networks, IJCNN 2019 [8852040] (Proceedings of the International Joint Conference on Neural Networks; Vol. 2019-July). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IJCNN.2019.8852040
Seong, Hongje ; Hyun, Junhyuk ; Chang, Hyunbae ; Lee, Suhyeon ; Woo, Suhan ; Kim, Euntai. / Scene Recognition via Object-to-Scene Class Conversion : End-to-End Training. 2019 International Joint Conference on Neural Networks, IJCNN 2019. Institute of Electrical and Electronics Engineers Inc., 2019. (Proceedings of the International Joint Conference on Neural Networks).
@inproceedings{bd1d19fea6d0420bbb11f9a904f1c306,
title = "Scene Recognition via Object-to-Scene Class Conversion: End-to-End Training",
abstract = "When a person recognize the scene of an image, contextual understanding from its environmental elements is necessary. These environmental elements are variant and require comprehensive understanding of various situations. Especially, objects are frequently used as environmental elements related with scene. In this paper, we suggest a score level Class Conversion Matrix (CCM) for scene recognition with a great focus on relationship between objects and scene. A lot of existing methods have already build scene recognition systems with consideration of close relationship between object and scenes. However, most of these methods are using the object features directly without any conversions or reconstructions, and it lack confirmation whether these object features are helpful to recognize scenes correctly. To solve this problem, CCM, a matrix converting object feature to scene feature, is suggested. Moreover, CCM can be implemented with neural network layer and end-to-end trainable. Extensive experiments on Places 2 dataset demonstrate the effectiveness of our approach, when it is applied to the existing deep convolutional neural network architectures. The code is available at https://github.com/Hongje/Class_Conversion_Matrix-Places365.",
author = "Hongje Seong and Junhyuk Hyun and Hyunbae Chang and Suhyeon Lee and Suhan Woo and Euntai Kim",
year = "2019",
month = "7",
doi = "10.1109/IJCNN.2019.8852040",
language = "English",
series = "Proceedings of the International Joint Conference on Neural Networks",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "2019 International Joint Conference on Neural Networks, IJCNN 2019",
address = "United States",

}

Seong, H, Hyun, J, Chang, H, Lee, S, Woo, S & Kim, E 2019, Scene Recognition via Object-to-Scene Class Conversion: End-to-End Training. in 2019 International Joint Conference on Neural Networks, IJCNN 2019., 8852040, Proceedings of the International Joint Conference on Neural Networks, vol. 2019-July, Institute of Electrical and Electronics Engineers Inc., 2019 International Joint Conference on Neural Networks, IJCNN 2019, Budapest, Hungary, 19/7/14. https://doi.org/10.1109/IJCNN.2019.8852040

Scene Recognition via Object-to-Scene Class Conversion : End-to-End Training. / Seong, Hongje; Hyun, Junhyuk; Chang, Hyunbae; Lee, Suhyeon; Woo, Suhan; Kim, Euntai.

2019 International Joint Conference on Neural Networks, IJCNN 2019. Institute of Electrical and Electronics Engineers Inc., 2019. 8852040 (Proceedings of the International Joint Conference on Neural Networks; Vol. 2019-July).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Scene Recognition via Object-to-Scene Class Conversion

T2 - End-to-End Training

AU - Seong, Hongje

AU - Hyun, Junhyuk

AU - Chang, Hyunbae

AU - Lee, Suhyeon

AU - Woo, Suhan

AU - Kim, Euntai

PY - 2019/7

Y1 - 2019/7

N2 - When a person recognize the scene of an image, contextual understanding from its environmental elements is necessary. These environmental elements are variant and require comprehensive understanding of various situations. Especially, objects are frequently used as environmental elements related with scene. In this paper, we suggest a score level Class Conversion Matrix (CCM) for scene recognition with a great focus on relationship between objects and scene. A lot of existing methods have already build scene recognition systems with consideration of close relationship between object and scenes. However, most of these methods are using the object features directly without any conversions or reconstructions, and it lack confirmation whether these object features are helpful to recognize scenes correctly. To solve this problem, CCM, a matrix converting object feature to scene feature, is suggested. Moreover, CCM can be implemented with neural network layer and end-to-end trainable. Extensive experiments on Places 2 dataset demonstrate the effectiveness of our approach, when it is applied to the existing deep convolutional neural network architectures. The code is available at https://github.com/Hongje/Class_Conversion_Matrix-Places365.

AB - When a person recognize the scene of an image, contextual understanding from its environmental elements is necessary. These environmental elements are variant and require comprehensive understanding of various situations. Especially, objects are frequently used as environmental elements related with scene. In this paper, we suggest a score level Class Conversion Matrix (CCM) for scene recognition with a great focus on relationship between objects and scene. A lot of existing methods have already build scene recognition systems with consideration of close relationship between object and scenes. However, most of these methods are using the object features directly without any conversions or reconstructions, and it lack confirmation whether these object features are helpful to recognize scenes correctly. To solve this problem, CCM, a matrix converting object feature to scene feature, is suggested. Moreover, CCM can be implemented with neural network layer and end-to-end trainable. Extensive experiments on Places 2 dataset demonstrate the effectiveness of our approach, when it is applied to the existing deep convolutional neural network architectures. The code is available at https://github.com/Hongje/Class_Conversion_Matrix-Places365.

UR - http://www.scopus.com/inward/record.url?scp=85073203784&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85073203784&partnerID=8YFLogxK

U2 - 10.1109/IJCNN.2019.8852040

DO - 10.1109/IJCNN.2019.8852040

M3 - Conference contribution

AN - SCOPUS:85073203784

T3 - Proceedings of the International Joint Conference on Neural Networks

BT - 2019 International Joint Conference on Neural Networks, IJCNN 2019

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Seong H, Hyun J, Chang H, Lee S, Woo S, Kim E. Scene Recognition via Object-to-Scene Class Conversion: End-to-End Training. In 2019 International Joint Conference on Neural Networks, IJCNN 2019. Institute of Electrical and Electronics Engineers Inc. 2019. 8852040. (Proceedings of the International Joint Conference on Neural Networks). https://doi.org/10.1109/IJCNN.2019.8852040