We present a data-efficient representation learning approach to learn video representation with small amount of labeled data. We propose a multitask learning model ActionFlowNet to train a single stream network directly from raw pixels to jointly estimate optical flow while recognizing actions with convolutional neural networks, capturing both appearance and motion in a single model. Our model effectively learns video representation from motion information on unlabeled videos. Our model significantly improves action recognition accuracy by a large margin (23.6%) compared to state-of-the-art CNN-based unsupervised representation learning methods trained without external large scale data and additional optical flow input. Without pretraining on large external labeled datasets, our model, by well exploiting the motion information, achieves competitive recognition accuracy to the models trained with large labeled datasets such as ImageNet and Sport-1M.
|Title of host publication||Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018|
|Publisher||Institute of Electrical and Electronics Engineers Inc.|
|Number of pages||9|
|Publication status||Published - 2018 May 3|
|Event||18th IEEE Winter Conference on Applications of Computer Vision, WACV 2018 - Lake Tahoe, United States|
Duration: 2018 Mar 12 → 2018 Mar 15
|Name||Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018|
|Conference||18th IEEE Winter Conference on Applications of Computer Vision, WACV 2018|
|Period||18/3/12 → 18/3/15|
Bibliographical notePublisher Copyright:
© 2018 IEEE.
All Science Journal Classification (ASJC) codes
- Computer Vision and Pattern Recognition
- Computer Science Applications