Multi-Task Learning Framework for Motion Estimation and Dynamic Scene Deblurring

Hyungjoo Jung, Youngjung Kim, Hyunsung Jang, Namkoo Ha, Kwanghoon Sohn

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)


Motion blur, which disturbs human and machine perceptions of a scene, has been considered an unnecessary artifact that should be removed. However, the blur can be a useful clue to understanding the dynamic scene, since various sources of motion generate different types of artifacts. Motivated by the relationship between motion and blur, we propose a motion-aware feature learning framework for dynamic scene deblurring through multi-task learning. Our multi-task framework simultaneously estimates a deblurred image and a motion field from a blurred image. We design the encoder-decoder architectures for two tasks, and the encoder part is shared between them. Our motion estimation network could effectively distinguish between different types of blur, which facilitates image deblurring. Understanding implicit motion information through image deblurring could improve the performance of motion estimation. In addition to sharing the network between two tasks, we propose a reblurring loss function to optimize the overall parameters in our multi-task architecture. We provide an intensive analysis of complementary tasks to show the effectiveness of our multi-task framework. Furthermore, the experimental results demonstrate that the proposed method outperforms the state-of-the-art deblurring methods with respect to both qualitative and quantitative evaluations.

Original languageEnglish
Pages (from-to)8170-8183
Number of pages14
JournalIEEE Transactions on Image Processing
Publication statusPublished - 2021

Bibliographical note

Funding Information:
Manuscript received August 10, 2020; revised February 25, 2021, June 11, 2021, and August 14, 2021; accepted August 16, 2021. Date of publication September 22, 2021; date of current version September 29, 2021. This research was supported by R&D program for Advanced Integrated-intelligence for Identification (AIID) through the National Research Foundation of KOREA (NRF) funded by Ministry of Science and ICT (NRF-2018M3E3A1057289) and in part by the Yonsei Univiersity Research Fund of 2021 (2021-22-0001). The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Zhen He. (Corresponding author: Kwanghoon Sohn.) Hyungjoo Jung is with the Artificial Intelligence and Robotics Institute, Korea Institute of Science and Technology, Seoul 02792, South Korea (e-mail:

Publisher Copyright:
© 1992-2012 IEEE.

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Multi-Task Learning Framework for Motion Estimation and Dynamic Scene Deblurring'. Together they form a unique fingerprint.

Cite this