Convolutional neural networks (CNNs) have recently been applied to the optical flow estimation problem. As training the CNNs requires sufficiently large amounts of labeled data, existing approaches resort to synthetic, unrealistic datasets. On the other hand, unsupervised methods are capable of leveraging real-world videos for training where the ground truth flow fields are not available. These methods, however, rely on the fundamental assumptions of brightness constancy and spatial smoothness priors that do not hold near motion boundaries. In this paper, we propose to exploit unlabeled videos for semi-supervised learning of optical flow with a Generative Adversarial Network. Our key insight is that the adversarial loss can capture the structural patterns of flow warp errors without making explicit assumptions. Extensive experiments on benchmark datasets demonstrate that the proposed semi-supervised algorithm performs favorably against purely supervised and baseline semi-supervised learning schemes.
|Number of pages||11|
|Journal||Advances in Neural Information Processing Systems|
|Publication status||Published - 2017|
|Event||31st Annual Conference on Neural Information Processing Systems, NIPS 2017 - Long Beach, United States|
Duration: 2017 Dec 4 → 2017 Dec 9
Bibliographical noteFunding Information:
This work is supported in part by the NSF CAREER Grant #1149783, gifts from Adobe and NVIDIA.
All Science Journal Classification (ASJC) codes
- Computer Networks and Communications
- Information Systems
- Signal Processing