The key to video inpainting is to use correlation information from as many reference frames as possible. Existing flow-based propagation methods split the video synthesis process into multiple steps: flow completion → pixel propagation → synthesis. However, there is a significant drawback that the errors in each step continue to accumulate and amplify in the next step. To this end, we propose an Error Compensation Framework for Flow-guided Video Inpainting (ECFVI), which takes advantage of the flow-based method and offsets its weaknesses. We address the weakness with the newly designed flow completion module and the error compensation network that exploits the error guidance map. Our approach greatly improves the temporal consistency and the visual quality of the completed videos. Experimental results show the superior performance of our proposed method with the speed up of × 6, compared to the state-of-the-art methods. In addition, we present a new benchmark dataset for evaluation by supplementing the weaknesses of existing test datasets.
|Title of host publication||Computer Vision – ECCV 2022 - 17th European Conference, 2022, Proceedings|
|Editors||Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, Tal Hassner|
|Publisher||Springer Science and Business Media Deutschland GmbH|
|Number of pages||16|
|Publication status||Published - 2022|
|Event||17th European Conference on Computer Vision, ECCV 2022 - Tel Aviv, Israel|
Duration: 2022 Oct 23 → 2022 Oct 27
|Name||Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)|
|Conference||17th European Conference on Computer Vision, ECCV 2022|
|Period||22/10/23 → 22/10/27|
Bibliographical noteFunding Information:
Acknowledgement. This work was supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2014-3-00123, Development of High Performance Visual BigData Discovery Platform for LargeScale Realtime Data Analysis), and No. 2020-0-01361, Artificial Intelligence Graduate School Program (Yonsei University).
© 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.
All Science Journal Classification (ASJC) codes
- Theoretical Computer Science
- Computer Science(all)