Fast video multi-style transfer

Wei Gao, Yijun Li, Yihang Yin, Ming Hsuan Yang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Recent progress in video style transfer has shown promising results which contain less flickering effects. However, existing algorithms mainly trade off generality for efficiency, i.e., constructing one network per style example, and often work for short video clips only. In this work, we propose a video multi-style transfer (VMST) framework which enables fast and multi-style video transfer within one single network. Specifically, we design a multi-instance normalization block (MIN-Block) to learn different style examples and two ConvLSTM modules to encourage the temporal consistency. The proposed algorithm is demonstrated to be able to generate temporally-consistent video transfer results in different styles while keeping each stylized frame visually pleasing. Extensive experimental results show that the proposed method performs favorably against single-style models and some post-processing techniques that alleviate the flickering issue. We achieve as many as 120 stylization effects in a single model and show results on long-term videos that consist of thousands of frames.

Original languageEnglish
Title of host publicationProceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3211-3219
Number of pages9
ISBN (Electronic)9781728165530
DOIs
Publication statusPublished - 2020 Mar
Event2020 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2020 - Snowmass Village, United States
Duration: 2020 Mar 12020 Mar 5

Publication series

NameProceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020

Conference

Conference2020 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2020
CountryUnited States
CitySnowmass Village
Period20/3/120/3/5

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Computer Vision and Pattern Recognition

Fingerprint Dive into the research topics of 'Fast video multi-style transfer'. Together they form a unique fingerprint.

  • Cite this

    Gao, W., Li, Y., Yin, Y., & Yang, M. H. (2020). Fast video multi-style transfer. In Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020 (pp. 3211-3219). [9093420] (Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/WACV45572.2020.9093420