Deep Batch-Normalized LSTM networks with Auxiliary classifier for Skeleton based Action Recognition

Sungwoo Jun, Yoonsik Choe

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Recently, Recurrent Neural Network (RNN) based approach presents good performance in skeleton based action recognition. Utilizing deep layers of RNN or Long-Short Term Memory (LSTM) has its on weakness when handling long-sequence data because of vanishing or exploding gradients through time. Batch Normalized LSTM (BN-LSTM) is able to give a solution for the problem, with the merit of converging faster in training. In contrast, when deeply layered, BN-LSTM structure shows slow convergence and worse accuracy. In this work, we analyze deep-layered BN-LSTM shows slower convergence in early training phase in training scheme and, finally we propose a deep BN-LSTM structure with auxiliary classifier that is able to converge faster and gives better results at skeleton based action recognition problem. Some experiment are conducted with Penn Action dataset and our own Computer Assembling Video dataset, we verified our proposal shows better results in skeleton based action recognition.

Original languageEnglish
Title of host publicationIEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages279-284
Number of pages6
ISBN (Electronic)9781728102474
DOIs
Publication statusPublished - 2018 Jul 2
Event3rd IEEE International Conference on Image Processing, Applications and Systems, IPAS 2018 - Sophia Antipolis, France
Duration: 2018 Dec 122018 Dec 14

Publication series

NameIEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018

Conference

Conference3rd IEEE International Conference on Image Processing, Applications and Systems, IPAS 2018
CountryFrance
CitySophia Antipolis
Period18/12/1218/12/14

Fingerprint

Recurrent neural networks
Classifiers
Experiments
Long short-term memory

All Science Journal Classification (ASJC) codes

  • Computer Vision and Pattern Recognition
  • Hardware and Architecture
  • Computer Graphics and Computer-Aided Design

Cite this

Jun, S., & Choe, Y. (2018). Deep Batch-Normalized LSTM networks with Auxiliary classifier for Skeleton based Action Recognition. In IEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018 (pp. 279-284). [8708878] (IEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IPAS.2018.8708878
Jun, Sungwoo ; Choe, Yoonsik. / Deep Batch-Normalized LSTM networks with Auxiliary classifier for Skeleton based Action Recognition. IEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 279-284 (IEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018).
@inproceedings{1d37b73879bd470499071440d18ef345,
title = "Deep Batch-Normalized LSTM networks with Auxiliary classifier for Skeleton based Action Recognition",
abstract = "Recently, Recurrent Neural Network (RNN) based approach presents good performance in skeleton based action recognition. Utilizing deep layers of RNN or Long-Short Term Memory (LSTM) has its on weakness when handling long-sequence data because of vanishing or exploding gradients through time. Batch Normalized LSTM (BN-LSTM) is able to give a solution for the problem, with the merit of converging faster in training. In contrast, when deeply layered, BN-LSTM structure shows slow convergence and worse accuracy. In this work, we analyze deep-layered BN-LSTM shows slower convergence in early training phase in training scheme and, finally we propose a deep BN-LSTM structure with auxiliary classifier that is able to converge faster and gives better results at skeleton based action recognition problem. Some experiment are conducted with Penn Action dataset and our own Computer Assembling Video dataset, we verified our proposal shows better results in skeleton based action recognition.",
author = "Sungwoo Jun and Yoonsik Choe",
year = "2018",
month = "7",
day = "2",
doi = "10.1109/IPAS.2018.8708878",
language = "English",
series = "IEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "279--284",
booktitle = "IEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018",
address = "United States",

}

Jun, S & Choe, Y 2018, Deep Batch-Normalized LSTM networks with Auxiliary classifier for Skeleton based Action Recognition. in IEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018., 8708878, IEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018, Institute of Electrical and Electronics Engineers Inc., pp. 279-284, 3rd IEEE International Conference on Image Processing, Applications and Systems, IPAS 2018, Sophia Antipolis, France, 18/12/12. https://doi.org/10.1109/IPAS.2018.8708878

Deep Batch-Normalized LSTM networks with Auxiliary classifier for Skeleton based Action Recognition. / Jun, Sungwoo; Choe, Yoonsik.

IEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 279-284 8708878 (IEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Deep Batch-Normalized LSTM networks with Auxiliary classifier for Skeleton based Action Recognition

AU - Jun, Sungwoo

AU - Choe, Yoonsik

PY - 2018/7/2

Y1 - 2018/7/2

N2 - Recently, Recurrent Neural Network (RNN) based approach presents good performance in skeleton based action recognition. Utilizing deep layers of RNN or Long-Short Term Memory (LSTM) has its on weakness when handling long-sequence data because of vanishing or exploding gradients through time. Batch Normalized LSTM (BN-LSTM) is able to give a solution for the problem, with the merit of converging faster in training. In contrast, when deeply layered, BN-LSTM structure shows slow convergence and worse accuracy. In this work, we analyze deep-layered BN-LSTM shows slower convergence in early training phase in training scheme and, finally we propose a deep BN-LSTM structure with auxiliary classifier that is able to converge faster and gives better results at skeleton based action recognition problem. Some experiment are conducted with Penn Action dataset and our own Computer Assembling Video dataset, we verified our proposal shows better results in skeleton based action recognition.

AB - Recently, Recurrent Neural Network (RNN) based approach presents good performance in skeleton based action recognition. Utilizing deep layers of RNN or Long-Short Term Memory (LSTM) has its on weakness when handling long-sequence data because of vanishing or exploding gradients through time. Batch Normalized LSTM (BN-LSTM) is able to give a solution for the problem, with the merit of converging faster in training. In contrast, when deeply layered, BN-LSTM structure shows slow convergence and worse accuracy. In this work, we analyze deep-layered BN-LSTM shows slower convergence in early training phase in training scheme and, finally we propose a deep BN-LSTM structure with auxiliary classifier that is able to converge faster and gives better results at skeleton based action recognition problem. Some experiment are conducted with Penn Action dataset and our own Computer Assembling Video dataset, we verified our proposal shows better results in skeleton based action recognition.

UR - http://www.scopus.com/inward/record.url?scp=85066298406&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066298406&partnerID=8YFLogxK

U2 - 10.1109/IPAS.2018.8708878

DO - 10.1109/IPAS.2018.8708878

M3 - Conference contribution

AN - SCOPUS:85066298406

T3 - IEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018

SP - 279

EP - 284

BT - IEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Jun S, Choe Y. Deep Batch-Normalized LSTM networks with Auxiliary classifier for Skeleton based Action Recognition. In IEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 279-284. 8708878. (IEEE 3rd International Conference on Image Processing, Applications and Systems, IPAS 2018). https://doi.org/10.1109/IPAS.2018.8708878