A simple and effective fusion approach for multi-frame optical flow estimation

Zhile Ren, Orazio Gallo, Deqing Sun, Ming Hsuan Yang, Erik B. Sudderth, Jan Kautz

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

To date, top-performing optical flow estimation methods only take pairs of consecutive frames into account. While elegant and appealing, the idea of using more than two frames has not yet produced state-of-the-art results. We present a simple, yet effective fusion approach for multi-frame optical flow that benefits from longer-term temporal cues. Our method first warps the optical flow from previous frames to the current, thereby yielding multiple plausible estimates. It then fuses the complementary information carried by these estimates into a new optical flow field. At the time of writing, our method ranks first among published results in the MPI Sintel and KITTI 2015 benchmarks.

Original languageEnglish
Title of host publicationComputer Vision – ECCV 2018 Workshops, Proceedings
EditorsLaura Leal-Taixé, Stefan Roth
PublisherSpringer Verlag
Pages706-710
Number of pages5
ISBN (Print)9783030110239
DOIs
Publication statusPublished - 2019 Jan 1
Event15th European Conference on Computer Vision, ECCV 2018 - Munich, Germany
Duration: 2018 Sep 82018 Sep 14

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11134 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other15th European Conference on Computer Vision, ECCV 2018
CountryGermany
CityMunich
Period18/9/818/9/14

Fingerprint

Optical flows
Optical Flow
Fusion
Fusion reactions
Electric fuses
Estimate
Flow Field
Consecutive
Flow fields
Benchmark
Term

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Ren, Z., Gallo, O., Sun, D., Yang, M. H., Sudderth, E. B., & Kautz, J. (2019). A simple and effective fusion approach for multi-frame optical flow estimation. In L. Leal-Taixé, & S. Roth (Eds.), Computer Vision – ECCV 2018 Workshops, Proceedings (pp. 706-710). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11134 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-11024-6_53
Ren, Zhile ; Gallo, Orazio ; Sun, Deqing ; Yang, Ming Hsuan ; Sudderth, Erik B. ; Kautz, Jan. / A simple and effective fusion approach for multi-frame optical flow estimation. Computer Vision – ECCV 2018 Workshops, Proceedings. editor / Laura Leal-Taixé ; Stefan Roth. Springer Verlag, 2019. pp. 706-710 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{20079b256dd74463a52f63364396c45b,
title = "A simple and effective fusion approach for multi-frame optical flow estimation",
abstract = "To date, top-performing optical flow estimation methods only take pairs of consecutive frames into account. While elegant and appealing, the idea of using more than two frames has not yet produced state-of-the-art results. We present a simple, yet effective fusion approach for multi-frame optical flow that benefits from longer-term temporal cues. Our method first warps the optical flow from previous frames to the current, thereby yielding multiple plausible estimates. It then fuses the complementary information carried by these estimates into a new optical flow field. At the time of writing, our method ranks first among published results in the MPI Sintel and KITTI 2015 benchmarks.",
author = "Zhile Ren and Orazio Gallo and Deqing Sun and Yang, {Ming Hsuan} and Sudderth, {Erik B.} and Jan Kautz",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/978-3-030-11024-6_53",
language = "English",
isbn = "9783030110239",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "706--710",
editor = "Laura Leal-Taix{\'e} and Stefan Roth",
booktitle = "Computer Vision – ECCV 2018 Workshops, Proceedings",
address = "Germany",

}

Ren, Z, Gallo, O, Sun, D, Yang, MH, Sudderth, EB & Kautz, J 2019, A simple and effective fusion approach for multi-frame optical flow estimation. in L Leal-Taixé & S Roth (eds), Computer Vision – ECCV 2018 Workshops, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11134 LNCS, Springer Verlag, pp. 706-710, 15th European Conference on Computer Vision, ECCV 2018, Munich, Germany, 18/9/8. https://doi.org/10.1007/978-3-030-11024-6_53

A simple and effective fusion approach for multi-frame optical flow estimation. / Ren, Zhile; Gallo, Orazio; Sun, Deqing; Yang, Ming Hsuan; Sudderth, Erik B.; Kautz, Jan.

Computer Vision – ECCV 2018 Workshops, Proceedings. ed. / Laura Leal-Taixé; Stefan Roth. Springer Verlag, 2019. p. 706-710 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11134 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - A simple and effective fusion approach for multi-frame optical flow estimation

AU - Ren, Zhile

AU - Gallo, Orazio

AU - Sun, Deqing

AU - Yang, Ming Hsuan

AU - Sudderth, Erik B.

AU - Kautz, Jan

PY - 2019/1/1

Y1 - 2019/1/1

N2 - To date, top-performing optical flow estimation methods only take pairs of consecutive frames into account. While elegant and appealing, the idea of using more than two frames has not yet produced state-of-the-art results. We present a simple, yet effective fusion approach for multi-frame optical flow that benefits from longer-term temporal cues. Our method first warps the optical flow from previous frames to the current, thereby yielding multiple plausible estimates. It then fuses the complementary information carried by these estimates into a new optical flow field. At the time of writing, our method ranks first among published results in the MPI Sintel and KITTI 2015 benchmarks.

AB - To date, top-performing optical flow estimation methods only take pairs of consecutive frames into account. While elegant and appealing, the idea of using more than two frames has not yet produced state-of-the-art results. We present a simple, yet effective fusion approach for multi-frame optical flow that benefits from longer-term temporal cues. Our method first warps the optical flow from previous frames to the current, thereby yielding multiple plausible estimates. It then fuses the complementary information carried by these estimates into a new optical flow field. At the time of writing, our method ranks first among published results in the MPI Sintel and KITTI 2015 benchmarks.

UR - http://www.scopus.com/inward/record.url?scp=85061740901&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061740901&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-11024-6_53

DO - 10.1007/978-3-030-11024-6_53

M3 - Conference contribution

AN - SCOPUS:85061740901

SN - 9783030110239

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 706

EP - 710

BT - Computer Vision – ECCV 2018 Workshops, Proceedings

A2 - Leal-Taixé, Laura

A2 - Roth, Stefan

PB - Springer Verlag

ER -

Ren Z, Gallo O, Sun D, Yang MH, Sudderth EB, Kautz J. A simple and effective fusion approach for multi-frame optical flow estimation. In Leal-Taixé L, Roth S, editors, Computer Vision – ECCV 2018 Workshops, Proceedings. Springer Verlag. 2019. p. 706-710. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-11024-6_53