A fusion approach for multi-frame optical flow estimation

Zhile Ren; Orazio Gallo; Deqing Sun; Ming Hsuan Yang; Erik B. Sudderth; Jan Kautz

doi:10.1109/WACV.2019.00225

A fusion approach for multi-frame optical flow estimation

Zhile Ren, Orazio Gallo, Deqing Sun, Ming Hsuan Yang, Erik B. Sudderth, Jan Kautz

College of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

40 Citations (Scopus)

Abstract

To date, top-performing optical flow estimation methods only take pairs of consecutive frames into account. While elegant and appealing, the idea of using more than two frames has not yet produced state-of-the-art results. We present a simple, yet effective fusion approach for multi-frame optical flow that benefits from longer-term temporal cues. Our method first warps the optical flow from previous frames to the current, thereby yielding multiple plausible estimates. It then fuses the complementary information carried by these estimates into a new optical flow field. At the time of writing, our method ranks first among published results in the MPI Sintel and KITTI 2015 benchmarks. Our models will be available on https://github.com/NVlabs/PWC-Net.

Original language	English
Title of host publication	Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	2077-2086
Number of pages	10
ISBN (Electronic)	9781728119755
DOIs	https://doi.org/10.1109/WACV.2019.00225
Publication status	Published - 2019 Mar 4
Event	19th IEEE Winter Conference on Applications of Computer Vision, WACV 2019 - Waikoloa Village, United States Duration: 2019 Jan 7 → 2019 Jan 11

Publication series

Name	Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019

Conference

Conference	19th IEEE Winter Conference on Applications of Computer Vision, WACV 2019
Country/Territory	United States
City	Waikoloa Village
Period	19/1/7 → 19/1/11

Bibliographical note

Publisher Copyright:
© 2019 IEEE

All Science Journal Classification (ASJC) codes

Computer Vision and Pattern Recognition
Computer Science Applications

Access to Document

10.1109/WACV.2019.00225

Cite this

Ren, Z., Gallo, O., Sun, D., Yang, M. H., Sudderth, E. B., & Kautz, J. (2019). A fusion approach for multi-frame optical flow estimation. In Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019 (pp. 2077-2086). [8658399] (Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/WACV.2019.00225

@inproceedings{8cab6bead50a4ce0bb51746a5deb3279,

title = "A fusion approach for multi-frame optical flow estimation",

abstract = "To date, top-performing optical flow estimation methods only take pairs of consecutive frames into account. While elegant and appealing, the idea of using more than two frames has not yet produced state-of-the-art results. We present a simple, yet effective fusion approach for multi-frame optical flow that benefits from longer-term temporal cues. Our method first warps the optical flow from previous frames to the current, thereby yielding multiple plausible estimates. It then fuses the complementary information carried by these estimates into a new optical flow field. At the time of writing, our method ranks first among published results in the MPI Sintel and KITTI 2015 benchmarks. Our models will be available on https://github.com/NVlabs/PWC-Net.",

author = "Zhile Ren and Orazio Gallo and Deqing Sun and Yang, {Ming Hsuan} and Sudderth, {Erik B.} and Jan Kautz",

note = "Publisher Copyright: {\textcopyright} 2019 IEEE; 19th IEEE Winter Conference on Applications of Computer Vision, WACV 2019 ; Conference date: 07-01-2019 Through 11-01-2019",

year = "2019",

month = mar,

day = "4",

doi = "10.1109/WACV.2019.00225",

language = "English",

series = "Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "2077--2086",

booktitle = "Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019",

address = "United States",

}

Ren, Z, Gallo, O, Sun, D, Yang, MH, Sudderth, EB & Kautz, J 2019, A fusion approach for multi-frame optical flow estimation. in Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019., 8658399, Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019, Institute of Electrical and Electronics Engineers Inc., pp. 2077-2086, 19th IEEE Winter Conference on Applications of Computer Vision, WACV 2019, Waikoloa Village, United States, 19/1/7. https://doi.org/10.1109/WACV.2019.00225

A fusion approach for multi-frame optical flow estimation. / Ren, Zhile; Gallo, Orazio; Sun, Deqing et al.

Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019. Institute of Electrical and Electronics Engineers Inc., 2019. p. 2077-2086 8658399 (Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - A fusion approach for multi-frame optical flow estimation

AU - Ren, Zhile

AU - Gallo, Orazio

AU - Sun, Deqing

AU - Yang, Ming Hsuan

AU - Sudderth, Erik B.

AU - Kautz, Jan

PY - 2019/3/4

Y1 - 2019/3/4

N2 - To date, top-performing optical flow estimation methods only take pairs of consecutive frames into account. While elegant and appealing, the idea of using more than two frames has not yet produced state-of-the-art results. We present a simple, yet effective fusion approach for multi-frame optical flow that benefits from longer-term temporal cues. Our method first warps the optical flow from previous frames to the current, thereby yielding multiple plausible estimates. It then fuses the complementary information carried by these estimates into a new optical flow field. At the time of writing, our method ranks first among published results in the MPI Sintel and KITTI 2015 benchmarks. Our models will be available on https://github.com/NVlabs/PWC-Net.

AB - To date, top-performing optical flow estimation methods only take pairs of consecutive frames into account. While elegant and appealing, the idea of using more than two frames has not yet produced state-of-the-art results. We present a simple, yet effective fusion approach for multi-frame optical flow that benefits from longer-term temporal cues. Our method first warps the optical flow from previous frames to the current, thereby yielding multiple plausible estimates. It then fuses the complementary information carried by these estimates into a new optical flow field. At the time of writing, our method ranks first among published results in the MPI Sintel and KITTI 2015 benchmarks. Our models will be available on https://github.com/NVlabs/PWC-Net.

UR - http://www.scopus.com/inward/record.url?scp=85063568956&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063568956&partnerID=8YFLogxK

U2 - 10.1109/WACV.2019.00225

DO - 10.1109/WACV.2019.00225

M3 - Conference contribution

AN - SCOPUS:85063568956

T3 - Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019

SP - 2077

EP - 2086

BT - Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 19th IEEE Winter Conference on Applications of Computer Vision, WACV 2019

Y2 - 7 January 2019 through 11 January 2019

ER -

Ren Z, Gallo O, Sun D, Yang MH, Sudderth EB, Kautz J. A fusion approach for multi-frame optical flow estimation. In Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019. Institute of Electrical and Electronics Engineers Inc. 2019. p. 2077-2086. 8658399. (Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019). doi: 10.1109/WACV.2019.00225

A fusion approach for multi-frame optical flow estimation

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this