Contextual action cues from camera sensor for multi-stream action recognition

Jongkwang Hong, Bora Cho, Yong Won Hong, Hyeran Byun

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)

Abstract

In action recognition research, two primary types of information are appearance and motion information that is learned from RGB images through visual sensors. However, depending on the action characteristics, contextual information, such as the existence of specific objects or globally-shared information in the image, becomes vital information to define the action. For example, the existence of the ball is vital information distinguishing “kicking” from “running”. Furthermore, some actions share typical global abstract poses, which can be used as a key to classify actions. Based on these observations, we propose the multi-stream network model, which incorporates spatial, temporal, and contextual cues in the image for action recognition. We experimented on the proposed method using C3D or inflated 3D ConvNet (I3D) as a backbone network, regarding two different action recognition datasets. As a result, we observed overall improvement in accuracy, demonstrating the effectiveness of our proposed method.

Original languageEnglish
Article number1382
JournalSensors (Switzerland)
Volume19
Issue number6
DOIs
Publication statusPublished - 2019 Mar 2

Bibliographical note

Funding Information:
Funding: This research was supported by the Next-Generation Information Computing Development Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT (NRF-2017M3C4A7069370).

All Science Journal Classification (ASJC) codes

  • Analytical Chemistry
  • Biochemistry
  • Atomic and Molecular Physics, and Optics
  • Instrumentation
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Contextual action cues from camera sensor for multi-stream action recognition'. Together they form a unique fingerprint.

Cite this