Video frame extrapolation is a task to predict future frames when the past frames are given. Unlike previous studies that usually have been focused on the design of modules or construction of networks, we propose a novel ExtrapolativeInterpolative Cycle (EIC) loss using pre-trained frame interpolation module to improve extrapolation performance. Cycle-consistency loss has been used for stable prediction between two function spaces in many visual tasks. We formulate this cycle-consistency using two mapping functions; frame extrapolation and interpolation. Since it is easier to predict intermediate frames than to predict future frames in terms of the object occlusion and motion uncertainty, interpolation module can give guidance signal effectively for training the extrapolation function. EIC loss can be applied to any existing extrapolation algorithms and guarantee consistent prediction in the short future as well as long future frames. Experimental results show that simply adding EIC loss to the existing baseline increases extrapolation performance on both UCF101  and KITTI  datasets.
|Title of host publication||2020 IEEE International Conference on Image Processing, ICIP 2020 - Proceedings|
|Publisher||IEEE Computer Society|
|Number of pages||5|
|Publication status||Published - 2020 Oct|
|Event||2020 IEEE International Conference on Image Processing, ICIP 2020 - Virtual, Abu Dhabi, United Arab Emirates|
Duration: 2020 Sept 25 → 2020 Sept 28
|Name||Proceedings - International Conference on Image Processing, ICIP|
|Conference||2020 IEEE International Conference on Image Processing, ICIP 2020|
|Country/Territory||United Arab Emirates|
|City||Virtual, Abu Dhabi|
|Period||20/9/25 → 20/9/28|
Bibliographical noteFunding Information:
The authors express their sincere gratitude to the Director and faculty members of NERIST forestry department for providing permission to carry out this work. We would also like to thank all forest officials of Manas NP and Rajiv Gandhi Orang NP for extending their valuable co-operation. We thank all people working in the Forestry department who directly or indirectly helped in successful completion of this work.
© 2020 IEEE.
All Science Journal Classification (ASJC) codes
- Computer Vision and Pattern Recognition
- Signal Processing