A simple and effective fusion approach for multi-frame optical flow estimation

Zhile Ren; Orazio Gallo; Deqing Sun; Ming Hsuan Yang; Erik B. Sudderth; Jan Kautz

doi:10.1007/978-3-030-11024-6_53

A simple and effective fusion approach for multi-frame optical flow estimation

Zhile Ren, Orazio Gallo, Deqing Sun, Ming Hsuan Yang, Erik B. Sudderth, Jan Kautz

Department of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

To date, top-performing optical flow estimation methods only take pairs of consecutive frames into account. While elegant and appealing, the idea of using more than two frames has not yet produced state-of-the-art results. We present a simple, yet effective fusion approach for multi-frame optical flow that benefits from longer-term temporal cues. Our method first warps the optical flow from previous frames to the current, thereby yielding multiple plausible estimates. It then fuses the complementary information carried by these estimates into a new optical flow field. At the time of writing, our method ranks first among published results in the MPI Sintel and KITTI 2015 benchmarks.

Original language	English
Title of host publication	Computer Vision – ECCV 2018 Workshops, Proceedings
Editors	Laura Leal-Taixé, Stefan Roth
Publisher	Springer Verlag
Pages	706-710
Number of pages	5
ISBN (Print)	9783030110239
DOIs	https://doi.org/10.1007/978-3-030-11024-6_53
Publication status	Published - 2019 Jan 1
Event	15th European Conference on Computer Vision, ECCV 2018 - Munich, Germany Duration: 2018 Sep 8 → 2018 Sep 14

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11134 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	15th European Conference on Computer Vision, ECCV 2018
Country	Germany
City	Munich
Period	18/9/8 → 18/9/14

Fingerprint

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
Computer Science(all)

Cite this

Ren, Z., Gallo, O., Sun, D., Yang, M. H., Sudderth, E. B., & Kautz, J. (2019). A simple and effective fusion approach for multi-frame optical flow estimation. In L. Leal-Taixé, & S. Roth (Eds.), Computer Vision – ECCV 2018 Workshops, Proceedings (pp. 706-710). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11134 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-11024-6_53

Ren, Zhile ; Gallo, Orazio ; Sun, Deqing ; Yang, Ming Hsuan ; Sudderth, Erik B. ; Kautz, Jan. / A simple and effective fusion approach for multi-frame optical flow estimation. Computer Vision – ECCV 2018 Workshops, Proceedings. editor / Laura Leal-Taixé ; Stefan Roth. Springer Verlag, 2019. pp. 706-710 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{20079b256dd74463a52f63364396c45b,

title = "A simple and effective fusion approach for multi-frame optical flow estimation",

abstract = "To date, top-performing optical flow estimation methods only take pairs of consecutive frames into account. While elegant and appealing, the idea of using more than two frames has not yet produced state-of-the-art results. We present a simple, yet effective fusion approach for multi-frame optical flow that benefits from longer-term temporal cues. Our method first warps the optical flow from previous frames to the current, thereby yielding multiple plausible estimates. It then fuses the complementary information carried by these estimates into a new optical flow field. At the time of writing, our method ranks first among published results in the MPI Sintel and KITTI 2015 benchmarks.",

author = "Zhile Ren and Orazio Gallo and Deqing Sun and Yang, {Ming Hsuan} and Sudderth, {Erik B.} and Jan Kautz",

year = "2019",

month = jan

day = "1",

doi = "10.1007/978-3-030-11024-6_53",

language = "English",

isbn = "9783030110239",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "706--710",

editor = "Laura Leal-Taix{\'e} and Stefan Roth",

booktitle = "Computer Vision – ECCV 2018 Workshops, Proceedings",

address = "Germany",

note = "15th European Conference on Computer Vision, ECCV 2018 ; Conference date: 08-09-2018 Through 14-09-2018",

}

Ren, Z, Gallo, O, Sun, D, Yang, MH, Sudderth, EB & Kautz, J 2019, A simple and effective fusion approach for multi-frame optical flow estimation. in L Leal-Taixé & S Roth (eds), Computer Vision – ECCV 2018 Workshops, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11134 LNCS, Springer Verlag, pp. 706-710, 15th European Conference on Computer Vision, ECCV 2018, Munich, Germany, 18/9/8. https://doi.org/10.1007/978-3-030-11024-6_53

A simple and effective fusion approach for multi-frame optical flow estimation. / Ren, Zhile; Gallo, Orazio; Sun, Deqing; Yang, Ming Hsuan; Sudderth, Erik B.; Kautz, Jan.

Computer Vision – ECCV 2018 Workshops, Proceedings. ed. / Laura Leal-Taixé; Stefan Roth. Springer Verlag, 2019. p. 706-710 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11134 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - A simple and effective fusion approach for multi-frame optical flow estimation

AU - Ren, Zhile

AU - Gallo, Orazio

AU - Sun, Deqing

AU - Yang, Ming Hsuan

AU - Sudderth, Erik B.

AU - Kautz, Jan

PY - 2019/1/1

Y1 - 2019/1/1

N2 - To date, top-performing optical flow estimation methods only take pairs of consecutive frames into account. While elegant and appealing, the idea of using more than two frames has not yet produced state-of-the-art results. We present a simple, yet effective fusion approach for multi-frame optical flow that benefits from longer-term temporal cues. Our method first warps the optical flow from previous frames to the current, thereby yielding multiple plausible estimates. It then fuses the complementary information carried by these estimates into a new optical flow field. At the time of writing, our method ranks first among published results in the MPI Sintel and KITTI 2015 benchmarks.

AB - To date, top-performing optical flow estimation methods only take pairs of consecutive frames into account. While elegant and appealing, the idea of using more than two frames has not yet produced state-of-the-art results. We present a simple, yet effective fusion approach for multi-frame optical flow that benefits from longer-term temporal cues. Our method first warps the optical flow from previous frames to the current, thereby yielding multiple plausible estimates. It then fuses the complementary information carried by these estimates into a new optical flow field. At the time of writing, our method ranks first among published results in the MPI Sintel and KITTI 2015 benchmarks.

UR - http://www.scopus.com/inward/record.url?scp=85061740901&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061740901&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-11024-6_53

DO - 10.1007/978-3-030-11024-6_53

M3 - Conference contribution

AN - SCOPUS:85061740901

SN - 9783030110239

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 706

EP - 710

BT - Computer Vision – ECCV 2018 Workshops, Proceedings

A2 - Leal-Taixé, Laura

A2 - Roth, Stefan

PB - Springer Verlag

T2 - 15th European Conference on Computer Vision, ECCV 2018

Y2 - 8 September 2018 through 14 September 2018

ER -

Ren Z, Gallo O, Sun D, Yang MH, Sudderth EB, Kautz J. A simple and effective fusion approach for multi-frame optical flow estimation. In Leal-Taixé L, Roth S, editors, Computer Vision – ECCV 2018 Workshops, Proceedings. Springer Verlag. 2019. p. 706-710. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-11024-6_53

Access to Document

10.1007/978-3-030-11024-6_53