Extrapolative-Interpolative Cycle-Consistency Learning for Video Frame Extrapolation

Sangjin Lee, Hyeongmin Lee, Taeoh Kim, Sangyoun Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)


Video frame extrapolation is a task to predict future frames when the past frames are given. Unlike previous studies that usually have been focused on the design of modules or construction of networks, we propose a novel ExtrapolativeInterpolative Cycle (EIC) loss using pre-trained frame interpolation module to improve extrapolation performance. Cycle-consistency loss has been used for stable prediction between two function spaces in many visual tasks. We formulate this cycle-consistency using two mapping functions; frame extrapolation and interpolation. Since it is easier to predict intermediate frames than to predict future frames in terms of the object occlusion and motion uncertainty, interpolation module can give guidance signal effectively for training the extrapolation function. EIC loss can be applied to any existing extrapolation algorithms and guarantee consistent prediction in the short future as well as long future frames. Experimental results show that simply adding EIC loss to the existing baseline increases extrapolation performance on both UCF101 [1] and KITTI [2] datasets.

Original languageEnglish
Title of host publication2020 IEEE International Conference on Image Processing, ICIP 2020 - Proceedings
PublisherIEEE Computer Society
Number of pages5
ISBN (Electronic)9781728163956
Publication statusPublished - 2020 Oct
Event2020 IEEE International Conference on Image Processing, ICIP 2020 - Virtual, Abu Dhabi, United Arab Emirates
Duration: 2020 Sept 252020 Sept 28

Publication series

NameProceedings - International Conference on Image Processing, ICIP
ISSN (Print)1522-4880


Conference2020 IEEE International Conference on Image Processing, ICIP 2020
Country/TerritoryUnited Arab Emirates
CityVirtual, Abu Dhabi

Bibliographical note

Funding Information:
The authors express their sincere gratitude to the Director and faculty members of NERIST forestry department for providing permission to carry out this work. We would also like to thank all forest officials of Manas NP and Rajiv Gandhi Orang NP for extending their valuable co-operation. We thank all people working in the Forestry department who directly or indirectly helped in successful completion of this work.

Publisher Copyright:
© 2020 IEEE.

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition
  • Signal Processing


Dive into the research topics of 'Extrapolative-Interpolative Cycle-Consistency Learning for Video Frame Extrapolation'. Together they form a unique fingerprint.

Cite this