A Closed-Form Solution to Photorealistic Image Stylization

Yijun Li; Ming Yu Liu; Xueting Li; Ming Hsuan Yang; Jan Kautz

doi:10.1007/978-3-030-01219-9_28

A Closed-Form Solution to Photorealistic Image Stylization

Yijun Li, Ming Yu Liu, Xueting Li, Ming Hsuan Yang, Jan Kautz

College of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

41 Citations (Scopus)

Abstract

Photorealistic image stylization concerns transferring style of a reference photo to a content photo with the constraint that the stylized photo should remain photorealistic. While several photorealistic image stylization methods exist, they tend to generate spatially inconsistent stylizations with noticeable artifacts. In this paper, we propose a method to address these issues. The proposed method consists of a stylization step and a smoothing step. While the stylization step transfers the style of the reference photo to the content photo, the smoothing step ensures spatially consistent stylizations. Each of the steps has a closed-form solution and can be computed efficiently. We conduct extensive experimental validations. The results show that the proposed method generates photorealistic stylization outputs that are more preferred by human subjects as compared to those by the competing methods while running much faster. Source code and additional results are available at https://github.com/NVIDIA/FastPhotoStyle.

Original language	English
Title of host publication	Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings
Editors	Vittorio Ferrari, Cristian Sminchisescu, Martial Hebert, Yair Weiss
Publisher	Springer Verlag
Pages	468-483
Number of pages	16
ISBN (Print)	9783030012182
DOIs	https://doi.org/10.1007/978-3-030-01219-9_28
Publication status	Published - 2018
Event	15th European Conference on Computer Vision, ECCV 2018 - Munich, Germany Duration: 2018 Sep 8 → 2018 Sep 14

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11207 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	15th European Conference on Computer Vision, ECCV 2018
Country/Territory	Germany
City	Munich
Period	18/9/8 → 18/9/14

Bibliographical note

Funding Information:
Partial support for this work was obtained from the Israel Science Foundation in the framework of the INREP project and from Israel Ministry of Science and Technology in the framework of the Israel-India cooperation program.

Publisher Copyright:
© 2018, Springer Nature Switzerland AG.

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-030-01219-9_28

Cite this

Li, Y., Liu, M. Y., Li, X., Yang, M. H., & Kautz, J. (2018). A Closed-Form Solution to Photorealistic Image Stylization. In V. Ferrari, C. Sminchisescu, M. Hebert, & Y. Weiss (Eds.), Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings (pp. 468-483). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11207 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-01219-9_28

Li, Yijun ; Liu, Ming Yu ; Li, Xueting et al. / A Closed-Form Solution to Photorealistic Image Stylization. Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. editor / Vittorio Ferrari ; Cristian Sminchisescu ; Martial Hebert ; Yair Weiss. Springer Verlag, 2018. pp. 468-483 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{192676fdd6ae40238fc9e5d60682f8ec,

title = "A Closed-Form Solution to Photorealistic Image Stylization",

abstract = "Photorealistic image stylization concerns transferring style of a reference photo to a content photo with the constraint that the stylized photo should remain photorealistic. While several photorealistic image stylization methods exist, they tend to generate spatially inconsistent stylizations with noticeable artifacts. In this paper, we propose a method to address these issues. The proposed method consists of a stylization step and a smoothing step. While the stylization step transfers the style of the reference photo to the content photo, the smoothing step ensures spatially consistent stylizations. Each of the steps has a closed-form solution and can be computed efficiently. We conduct extensive experimental validations. The results show that the proposed method generates photorealistic stylization outputs that are more preferred by human subjects as compared to those by the competing methods while running much faster. Source code and additional results are available at https://github.com/NVIDIA/FastPhotoStyle.",

author = "Yijun Li and Liu, {Ming Yu} and Xueting Li and Yang, {Ming Hsuan} and Jan Kautz",

note = "Funding Information: Partial support for this work was obtained from the Israel Science Foundation in the framework of the INREP project and from Israel Ministry of Science and Technology in the framework of the Israel-India cooperation program. Publisher Copyright: {\textcopyright} 2018, Springer Nature Switzerland AG.; 15th European Conference on Computer Vision, ECCV 2018 ; Conference date: 08-09-2018 Through 14-09-2018",

year = "2018",

doi = "10.1007/978-3-030-01219-9_28",

language = "English",

isbn = "9783030012182",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "468--483",

editor = "Vittorio Ferrari and Cristian Sminchisescu and Martial Hebert and Yair Weiss",

booktitle = "Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings",

address = "Germany",

}

Li, Y, Liu, MY, Li, X, Yang, MH & Kautz, J 2018, A Closed-Form Solution to Photorealistic Image Stylization. in V Ferrari, C Sminchisescu, M Hebert & Y Weiss (eds), Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11207 LNCS, Springer Verlag, pp. 468-483, 15th European Conference on Computer Vision, ECCV 2018, Munich, Germany, 18/9/8. https://doi.org/10.1007/978-3-030-01219-9_28

A Closed-Form Solution to Photorealistic Image Stylization. / Li, Yijun; Liu, Ming Yu; Li, Xueting et al.

Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. ed. / Vittorio Ferrari; Cristian Sminchisescu; Martial Hebert; Yair Weiss. Springer Verlag, 2018. p. 468-483 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11207 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - A Closed-Form Solution to Photorealistic Image Stylization

AU - Li, Yijun

AU - Liu, Ming Yu

AU - Li, Xueting

AU - Yang, Ming Hsuan

AU - Kautz, Jan

N1 - Funding Information: Partial support for this work was obtained from the Israel Science Foundation in the framework of the INREP project and from Israel Ministry of Science and Technology in the framework of the Israel-India cooperation program. Publisher Copyright: © 2018, Springer Nature Switzerland AG.

PY - 2018

Y1 - 2018

N2 - Photorealistic image stylization concerns transferring style of a reference photo to a content photo with the constraint that the stylized photo should remain photorealistic. While several photorealistic image stylization methods exist, they tend to generate spatially inconsistent stylizations with noticeable artifacts. In this paper, we propose a method to address these issues. The proposed method consists of a stylization step and a smoothing step. While the stylization step transfers the style of the reference photo to the content photo, the smoothing step ensures spatially consistent stylizations. Each of the steps has a closed-form solution and can be computed efficiently. We conduct extensive experimental validations. The results show that the proposed method generates photorealistic stylization outputs that are more preferred by human subjects as compared to those by the competing methods while running much faster. Source code and additional results are available at https://github.com/NVIDIA/FastPhotoStyle.

AB - Photorealistic image stylization concerns transferring style of a reference photo to a content photo with the constraint that the stylized photo should remain photorealistic. While several photorealistic image stylization methods exist, they tend to generate spatially inconsistent stylizations with noticeable artifacts. In this paper, we propose a method to address these issues. The proposed method consists of a stylization step and a smoothing step. While the stylization step transfers the style of the reference photo to the content photo, the smoothing step ensures spatially consistent stylizations. Each of the steps has a closed-form solution and can be computed efficiently. We conduct extensive experimental validations. The results show that the proposed method generates photorealistic stylization outputs that are more preferred by human subjects as compared to those by the competing methods while running much faster. Source code and additional results are available at https://github.com/NVIDIA/FastPhotoStyle.

UR - http://www.scopus.com/inward/record.url?scp=85055104113&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85055104113&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-01219-9_28

DO - 10.1007/978-3-030-01219-9_28

M3 - Conference contribution

AN - SCOPUS:85055104113

SN - 9783030012182

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 468

EP - 483

BT - Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings

A2 - Ferrari, Vittorio

A2 - Sminchisescu, Cristian

A2 - Hebert, Martial

A2 - Weiss, Yair

PB - Springer Verlag

T2 - 15th European Conference on Computer Vision, ECCV 2018

Y2 - 8 September 2018 through 14 September 2018

ER -

Li Y, Liu MY, Li X, Yang MH, Kautz J. A Closed-Form Solution to Photorealistic Image Stylization. In Ferrari V, Sminchisescu C, Hebert M, Weiss Y, editors, Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. Springer Verlag. 2018. p. 468-483. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-01219-9_28

A Closed-Form Solution to Photorealistic Image Stylization

Abstract

Publication series

Other

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this