Joint Face Hallucination and Deblurring via Structure Generation and Detail Enhancement

Yibing Song, Jiawei Zhang, Lijun Gong, Shengfeng He, Linchao Bao, Jinshan Pan, Qingxiong Yang, Ming Hsuan Yang

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

We address the problem of restoring a high-resolution face image from a blurry low-resolution input. This problem is difficult as super-resolution and deblurring need to be tackled simultaneously. Moreover, existing algorithms cannot handle face images well as low-resolution face images do not have much texture which is especially critical for deblurring. In this paper, we propose an effective algorithm by utilizing the domain-specific knowledge of human faces to recover high-quality faces. We first propose a facial component guided deep Convolutional Neural Network (CNN) to restore a coarse face image, which is denoted as the base image where the facial component is automatically generated from the input face image. However, the CNN based method cannot handle image details well. We further develop a novel exemplar-based detail enhancement algorithm via facial component matching. Extensive experiments show that the proposed method outperforms the state-of-the-art algorithms both quantitatively and qualitatively.

Original languageEnglish
Pages (from-to)785-800
Number of pages16
JournalInternational Journal of Computer Vision
Volume127
Issue number6-7
DOIs
Publication statusPublished - 2019 Jun 1

Fingerprint

Neural networks
Textures
Experiments

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Cite this

Song, Yibing ; Zhang, Jiawei ; Gong, Lijun ; He, Shengfeng ; Bao, Linchao ; Pan, Jinshan ; Yang, Qingxiong ; Yang, Ming Hsuan. / Joint Face Hallucination and Deblurring via Structure Generation and Detail Enhancement. In: International Journal of Computer Vision. 2019 ; Vol. 127, No. 6-7. pp. 785-800.
@article{34656dd74b3b419e917c11e277a0a634,
title = "Joint Face Hallucination and Deblurring via Structure Generation and Detail Enhancement",
abstract = "We address the problem of restoring a high-resolution face image from a blurry low-resolution input. This problem is difficult as super-resolution and deblurring need to be tackled simultaneously. Moreover, existing algorithms cannot handle face images well as low-resolution face images do not have much texture which is especially critical for deblurring. In this paper, we propose an effective algorithm by utilizing the domain-specific knowledge of human faces to recover high-quality faces. We first propose a facial component guided deep Convolutional Neural Network (CNN) to restore a coarse face image, which is denoted as the base image where the facial component is automatically generated from the input face image. However, the CNN based method cannot handle image details well. We further develop a novel exemplar-based detail enhancement algorithm via facial component matching. Extensive experiments show that the proposed method outperforms the state-of-the-art algorithms both quantitatively and qualitatively.",
author = "Yibing Song and Jiawei Zhang and Lijun Gong and Shengfeng He and Linchao Bao and Jinshan Pan and Qingxiong Yang and Yang, {Ming Hsuan}",
year = "2019",
month = "6",
day = "1",
doi = "10.1007/s11263-019-01148-6",
language = "English",
volume = "127",
pages = "785--800",
journal = "International Journal of Computer Vision",
issn = "0920-5691",
publisher = "Springer Netherlands",
number = "6-7",

}

Joint Face Hallucination and Deblurring via Structure Generation and Detail Enhancement. / Song, Yibing; Zhang, Jiawei; Gong, Lijun; He, Shengfeng; Bao, Linchao; Pan, Jinshan; Yang, Qingxiong; Yang, Ming Hsuan.

In: International Journal of Computer Vision, Vol. 127, No. 6-7, 01.06.2019, p. 785-800.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Joint Face Hallucination and Deblurring via Structure Generation and Detail Enhancement

AU - Song, Yibing

AU - Zhang, Jiawei

AU - Gong, Lijun

AU - He, Shengfeng

AU - Bao, Linchao

AU - Pan, Jinshan

AU - Yang, Qingxiong

AU - Yang, Ming Hsuan

PY - 2019/6/1

Y1 - 2019/6/1

N2 - We address the problem of restoring a high-resolution face image from a blurry low-resolution input. This problem is difficult as super-resolution and deblurring need to be tackled simultaneously. Moreover, existing algorithms cannot handle face images well as low-resolution face images do not have much texture which is especially critical for deblurring. In this paper, we propose an effective algorithm by utilizing the domain-specific knowledge of human faces to recover high-quality faces. We first propose a facial component guided deep Convolutional Neural Network (CNN) to restore a coarse face image, which is denoted as the base image where the facial component is automatically generated from the input face image. However, the CNN based method cannot handle image details well. We further develop a novel exemplar-based detail enhancement algorithm via facial component matching. Extensive experiments show that the proposed method outperforms the state-of-the-art algorithms both quantitatively and qualitatively.

AB - We address the problem of restoring a high-resolution face image from a blurry low-resolution input. This problem is difficult as super-resolution and deblurring need to be tackled simultaneously. Moreover, existing algorithms cannot handle face images well as low-resolution face images do not have much texture which is especially critical for deblurring. In this paper, we propose an effective algorithm by utilizing the domain-specific knowledge of human faces to recover high-quality faces. We first propose a facial component guided deep Convolutional Neural Network (CNN) to restore a coarse face image, which is denoted as the base image where the facial component is automatically generated from the input face image. However, the CNN based method cannot handle image details well. We further develop a novel exemplar-based detail enhancement algorithm via facial component matching. Extensive experiments show that the proposed method outperforms the state-of-the-art algorithms both quantitatively and qualitatively.

UR - http://www.scopus.com/inward/record.url?scp=85060258792&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85060258792&partnerID=8YFLogxK

U2 - 10.1007/s11263-019-01148-6

DO - 10.1007/s11263-019-01148-6

M3 - Article

AN - SCOPUS:85060258792

VL - 127

SP - 785

EP - 800

JO - International Journal of Computer Vision

JF - International Journal of Computer Vision

SN - 0920-5691

IS - 6-7

ER -