Learning to tell brake and turn signals in videos using CNN-LSTM structure

Han Kai Hsu, Yi Hsuan Tsai, Xue Mei, Kuan Hui Lee, Naoki Nagasaka, Danil Prokhorov, Ming Hsuan Yang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We present a method that learns to tell rear signals from a number of frames using a deep learning framework. The proposed framework extracts spatial features with a convolution neural network (CNN), and then applies a long short term memory (LSTM) network to learn the long-term dependencies. The brake signal classifier is trained using RGB frames, while the turn signal is recognized via a two-step localization approach. The two separate classifiers are learned to recognize the static brake signals and the dynamic turn signals. As a result, our recognition system can recognize 8 different rear signals via the combined two classifiers in real-world traffic scenes. Experimental results show that our method is able to obtain more accurate predictions than using only the CNN to classify rear signals with time sequence inputs.

Original languageEnglish
Title of host publication2017 IEEE 20th International Conference on Intelligent Transportation Systems, ITSC 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1-6
Number of pages6
ISBN (Electronic)9781538615256
DOIs
Publication statusPublished - 2018 Mar 14
Event20th IEEE International Conference on Intelligent Transportation Systems, ITSC 2017 - Yokohama, Kanagawa, Japan
Duration: 2017 Oct 162017 Oct 19

Publication series

NameIEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC
Volume2018-March

Conference

Conference20th IEEE International Conference on Intelligent Transportation Systems, ITSC 2017
CountryJapan
CityYokohama, Kanagawa
Period17/10/1617/10/19

All Science Journal Classification (ASJC) codes

  • Automotive Engineering
  • Mechanical Engineering
  • Computer Science Applications

Fingerprint Dive into the research topics of 'Learning to tell brake and turn signals in videos using CNN-LSTM structure'. Together they form a unique fingerprint.

  • Cite this

    Hsu, H. K., Tsai, Y. H., Mei, X., Lee, K. H., Nagasaka, N., Prokhorov, D., & Yang, M. H. (2018). Learning to tell brake and turn signals in videos using CNN-LSTM structure. In 2017 IEEE 20th International Conference on Intelligent Transportation Systems, ITSC 2017 (pp. 1-6). (IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC; Vol. 2018-March). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ITSC.2017.8317782