Video summarization by learning relationships between action and scene

Jungin Park; Jiyoung Lee; Sangryul Jeon; Kwanghoon Sohn

doi:10.1109/ICCVW.2019.00193

Video summarization by learning relationships between action and scene

Jungin Park, Jiyoung Lee, Sangryul Jeon, Kwanghoon Sohn

Department of Electrical and Electronic Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

Abstract

We propose a novel deep architecture for video summarization in untrimmed videos that simultaneously recognizes action and scene classes for every video segments. Our networks accomplish this through a multi-task fusion approach based on two types of attention modules to explore semantic correlations between action and scene in the videos. The proposed networks consist of the feature embedding networks and attention inference networks to stochastically leverage the inferred action and scene feature representations. Additionally, we design a new center loss function that learns the feature representations by enforcing to minimize the intra-class variations and to maximize the inter-class variations. Our model achieves a score of 0.8409 for summarization and accuracy of 0.7294 for action and scene recognition on test set of CoVieW'19 dataset, which is ranked 3rd.

Original language	English
Title of host publication	Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	1545-1552
Number of pages	8
ISBN (Electronic)	9781728150239
DOIs	https://doi.org/10.1109/ICCVW.2019.00193
Publication status	Published - 2019 Oct
Event	17th IEEE/CVF International Conference on Computer Vision Workshop, ICCVW 2019 - Seoul, Korea, Republic of Duration: 2019 Oct 27 → 2019 Oct 28

Publication series

Name	Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019

Conference

Conference	17th IEEE/CVF International Conference on Computer Vision Workshop, ICCVW 2019
Country/Territory	Korea, Republic of
City	Seoul
Period	19/10/27 → 19/10/28

Bibliographical note

Funding Information:
This research was supported by Next-Generation Information Computing Development Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Science and ICT (NRF-2017M3C4A7069370).

Publisher Copyright:
© 2019 IEEE.

All Science Journal Classification (ASJC) codes

Computer Science Applications
Computer Vision and Pattern Recognition

Access to Document

10.1109/ICCVW.2019.00193

Cite this

Park, J., Lee, J., Jeon, S., & Sohn, K. (2019). Video summarization by learning relationships between action and scene. In Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019 (pp. 1545-1552). [9022169] (Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCVW.2019.00193

Park, Jungin ; Lee, Jiyoung ; Jeon, Sangryul ; Sohn, Kwanghoon. / Video summarization by learning relationships between action and scene. Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019. Institute of Electrical and Electronics Engineers Inc., 2019. pp. 1545-1552 (Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019).

@inproceedings{2d1af0064f07437ead01dc7f0b2a8cc7,

title = "Video summarization by learning relationships between action and scene",

abstract = "We propose a novel deep architecture for video summarization in untrimmed videos that simultaneously recognizes action and scene classes for every video segments. Our networks accomplish this through a multi-task fusion approach based on two types of attention modules to explore semantic correlations between action and scene in the videos. The proposed networks consist of the feature embedding networks and attention inference networks to stochastically leverage the inferred action and scene feature representations. Additionally, we design a new center loss function that learns the feature representations by enforcing to minimize the intra-class variations and to maximize the inter-class variations. Our model achieves a score of 0.8409 for summarization and accuracy of 0.7294 for action and scene recognition on test set of CoVieW'19 dataset, which is ranked 3rd.",

author = "Jungin Park and Jiyoung Lee and Sangryul Jeon and Kwanghoon Sohn",

note = "Funding Information: This research was supported by Next-Generation Information Computing Development Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Science and ICT (NRF-2017M3C4A7069370). Publisher Copyright: {\textcopyright} 2019 IEEE.; 17th IEEE/CVF International Conference on Computer Vision Workshop, ICCVW 2019 ; Conference date: 27-10-2019 Through 28-10-2019",

year = "2019",

month = oct,

doi = "10.1109/ICCVW.2019.00193",

language = "English",

series = "Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "1545--1552",

booktitle = "Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019",

address = "United States",

}

Park, J, Lee, J, Jeon, S & Sohn, K 2019, Video summarization by learning relationships between action and scene. in Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019., 9022169, Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019, Institute of Electrical and Electronics Engineers Inc., pp. 1545-1552, 17th IEEE/CVF International Conference on Computer Vision Workshop, ICCVW 2019, Seoul, Korea, Republic of, 19/10/27. https://doi.org/10.1109/ICCVW.2019.00193

Video summarization by learning relationships between action and scene. / Park, Jungin; Lee, Jiyoung; Jeon, Sangryul; Sohn, Kwanghoon.

Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 1545-1552 9022169 (Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Video summarization by learning relationships between action and scene

AU - Park, Jungin

AU - Lee, Jiyoung

AU - Jeon, Sangryul

AU - Sohn, Kwanghoon

N1 - Funding Information: This research was supported by Next-Generation Information Computing Development Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Science and ICT (NRF-2017M3C4A7069370). Publisher Copyright: © 2019 IEEE.

PY - 2019/10

Y1 - 2019/10

N2 - We propose a novel deep architecture for video summarization in untrimmed videos that simultaneously recognizes action and scene classes for every video segments. Our networks accomplish this through a multi-task fusion approach based on two types of attention modules to explore semantic correlations between action and scene in the videos. The proposed networks consist of the feature embedding networks and attention inference networks to stochastically leverage the inferred action and scene feature representations. Additionally, we design a new center loss function that learns the feature representations by enforcing to minimize the intra-class variations and to maximize the inter-class variations. Our model achieves a score of 0.8409 for summarization and accuracy of 0.7294 for action and scene recognition on test set of CoVieW'19 dataset, which is ranked 3rd.

AB - We propose a novel deep architecture for video summarization in untrimmed videos that simultaneously recognizes action and scene classes for every video segments. Our networks accomplish this through a multi-task fusion approach based on two types of attention modules to explore semantic correlations between action and scene in the videos. The proposed networks consist of the feature embedding networks and attention inference networks to stochastically leverage the inferred action and scene feature representations. Additionally, we design a new center loss function that learns the feature representations by enforcing to minimize the intra-class variations and to maximize the inter-class variations. Our model achieves a score of 0.8409 for summarization and accuracy of 0.7294 for action and scene recognition on test set of CoVieW'19 dataset, which is ranked 3rd.

UR - http://www.scopus.com/inward/record.url?scp=85082457559&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85082457559&partnerID=8YFLogxK

U2 - 10.1109/ICCVW.2019.00193

DO - 10.1109/ICCVW.2019.00193

M3 - Conference contribution

AN - SCOPUS:85082457559

T3 - Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019

SP - 1545

EP - 1552

BT - Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 17th IEEE/CVF International Conference on Computer Vision Workshop, ICCVW 2019

Y2 - 27 October 2019 through 28 October 2019

ER -

Park J, Lee J, Jeon S, Sohn K. Video summarization by learning relationships between action and scene. In Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 1545-1552. 9022169. (Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019). https://doi.org/10.1109/ICCVW.2019.00193

Video summarization by learning relationships between action and scene

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this