We present a novel deep learning based algorithm for video inpainting. Video inpainting is a process of completing corrupted or missing regions in videos. Video inpainting has additional challenges compared to image inpainting due to the extra temporal information as well as the need for maintaining the temporal coherency. We propose a novel DNN-based framework called the Copy-and-Paste Networks for video inpainting that takes advantage of additional information in other frames of the video. The network is trained to copy corresponding contents in reference frames and paste them to fill the holes in the target frame. Our network also includes an alignment network that computes homographies between frames for the alignment, enabling the network to take information from more distant frames for robustness. Our method produces visually pleasing and temporally coherent results while running faster than the state-of-the-art optimization-based method. In addition, we extend our framework for enhancing over/under exposed frames in videos. Using this enhancement technique, we were able to significantly improve the lane detection accuracy on road videos.
|Title of host publication||Proceedings - 2019 International Conference on Computer Vision, ICCV 2019|
|Publisher||Institute of Electrical and Electronics Engineers Inc.|
|Number of pages||9|
|Publication status||Published - 2019 Oct|
|Event||17th IEEE/CVF International Conference on Computer Vision, ICCV 2019 - Seoul, Korea, Republic of|
Duration: 2019 Oct 27 → 2019 Nov 2
|Name||Proceedings of the IEEE International Conference on Computer Vision|
|Conference||17th IEEE/CVF International Conference on Computer Vision, ICCV 2019|
|Country/Territory||Korea, Republic of|
|Period||19/10/27 → 19/11/2|
Bibliographical noteFunding Information:
This work was supported by Institute for Information & communication Technology Promotion (IITP) grant funded by the Korea government (MSIP) (2018-0-01858).
© 2019 IEEE.
All Science Journal Classification (ASJC) codes
- Computer Vision and Pattern Recognition