Motion Sickness Prediction in Stereoscopic Videos using 3D Convolutional Neural Networks

Tae Min Lee, Jong Chul Yoon, In Kwon Lee

Research output: Contribution to journalArticlepeer-review

9 Citations (Scopus)

Abstract

In this paper, we propose a three-dimensional (3D) convolutional neural network (CNN)-based method for predicting the degree of motion sickness induced by a 360° stereoscopic video. We consider the user's eye movement as a new feature, in addition to the motion velocity and depth features of a video used in previous work. For this purpose, we use saliency, optical flow, and disparity maps of an input video, which represent eye movement, velocity, and depth, respectively, as the input of the 3D CNN. To train our machine-learning model, we extend the dataset established in the previous work using two data augmentation techniques: frame shifting and pixel shifting. Consequently, our model can predict the degree of motion sickness more precisely than the previous method, and the results have a more similar correlation to the distribution of ground-truth sickness.

Original languageEnglish
Article number8642906
Pages (from-to)1919-1927
Number of pages9
JournalIEEE Transactions on Visualization and Computer Graphics
Volume25
Issue number5
DOIs
Publication statusPublished - 2019 May

Bibliographical note

Funding Information:
This work has supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT)(No. NRF-2017R1A2B4005469), and by the MSIT(Ministry of Science and ICT), Korea, under the ITRC(Information Technology Research Center) support program(IITP-2018-0-01419) supervised by the IITP(Institute for Information communications Technology Promotion.

All Science Journal Classification (ASJC) codes

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design

Fingerprint Dive into the research topics of 'Motion Sickness Prediction in Stereoscopic Videos using 3D Convolutional Neural Networks'. Together they form a unique fingerprint.

Cite this