SegFlow: Joint Learning for Video Object Segmentation and Optical Flow

Jingchun Cheng, Yi Hsuan Tsai, Shengjin Wang, Ming Hsuan Yang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

117 Citations (Scopus)

Abstract

This paper proposes an end-to-end trainable network, SegFlow, for simultaneously predicting pixel-wise object segmentation and optical flow in videos. The proposed SegFlow has two branches where useful information of object segmentation and optical flow is propagated bidirectionally in a unified framework. The segmentation branch is based on a fully convolutional network, which has been proved effective in image segmentation task, and the optical flow branch takes advantage of the FlowNet model. The unified framework is trained iteratively offline to learn a generic notion, and fine-tuned online for specific objects. Extensive experiments on both the video object segmentation and optical flow datasets demonstrate that introducing optical flow improves the performance of segmentation and vice versa, against the state-of-the-art algorithms.

Original languageEnglish
Title of host publicationProceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages686-695
Number of pages10
ISBN (Electronic)9781538610329
DOIs
Publication statusPublished - 2017 Dec 22
Event16th IEEE International Conference on Computer Vision, ICCV 2017 - Venice, Italy
Duration: 2017 Oct 222017 Oct 29

Publication series

NameProceedings of the IEEE International Conference on Computer Vision
Volume2017-October
ISSN (Print)1550-5499

Other

Other16th IEEE International Conference on Computer Vision, ICCV 2017
CountryItaly
CityVenice
Period17/10/2217/10/29

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition

Fingerprint Dive into the research topics of 'SegFlow: Joint Learning for Video Object Segmentation and Optical Flow'. Together they form a unique fingerprint.

  • Cite this

    Cheng, J., Tsai, Y. H., Wang, S., & Yang, M. H. (2017). SegFlow: Joint Learning for Video Object Segmentation and Optical Flow. In Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017 (pp. 686-695). [8237343] (Proceedings of the IEEE International Conference on Computer Vision; Vol. 2017-October). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCV.2017.81