Learning to detect, associate, and recognize human actions and surrounding scenes in untrimmed videos

Jungin Park, Jiyoung Lee, Sangryul Jeon, Sunok Kim, Seungryong Kim, Kwanghoon Sohn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

While recognizing human actions and surrounding scenes addresses different aspects of video understanding, they have strong correlations that can be used to complement the singular information of each other. In this paper, we propose an approach for joint action and scene recognition that is formulated in an end-to-end learning framework based on temporal attention techniques and the fusion of them. By applying temporal attention modules to the generic feature network, action and scene features are extracted efficiently, and then they are composed to a single feature vector through the proposed fusion module. Our experiments on the CoVieW18 dataset show that our model is able to detect temporal attention with only weak supervision, and remarkably improves multi-task action and scene classification accuracies.

Original languageEnglish
Title of host publicationCoVieW 2018 - Proceedings of the 1st Workshop and Challenge on Comprehensive Video Understanding in the Wild, co-located with MM 2018
PublisherAssociation for Computing Machinery, Inc
Pages21-26
Number of pages6
ISBN (Electronic)9781450359764
DOIs
Publication statusPublished - 2018 Oct 15
Event1st Workshop and Challenge on Comprehensive Video Understanding in the Wild, CoVieW 2018, in conjunction with ACM Multimedia, MM 2018 - Seoul, Korea, Republic of
Duration: 2018 Oct 22 → …

Publication series

NameCoVieW 2018 - Proceedings of the 1st Workshop and Challenge on Comprehensive Video Understanding in the Wild, co-located with MM 2018

Other

Other1st Workshop and Challenge on Comprehensive Video Understanding in the Wild, CoVieW 2018, in conjunction with ACM Multimedia, MM 2018
CountryKorea, Republic of
CitySeoul
Period18/10/22 → …

    Fingerprint

All Science Journal Classification (ASJC) codes

  • Computer Science(all)
  • Health Informatics
  • Media Technology

Cite this

Park, J., Lee, J., Jeon, S., Kim, S., Kim, S., & Sohn, K. (2018). Learning to detect, associate, and recognize human actions and surrounding scenes in untrimmed videos. In CoVieW 2018 - Proceedings of the 1st Workshop and Challenge on Comprehensive Video Understanding in the Wild, co-located with MM 2018 (pp. 21-26). (CoVieW 2018 - Proceedings of the 1st Workshop and Challenge on Comprehensive Video Understanding in the Wild, co-located with MM 2018). Association for Computing Machinery, Inc. https://doi.org/10.1145/3265987.3265989