FurryGAN: High Quality Foreground-Aware Image Synthesis

Jeongmin Bae; Mingi Kwon; Youngjung Uh

doi:10.1007/978-3-031-19781-9_40

FurryGAN: High Quality Foreground-Aware Image Synthesis

Jeongmin Bae, Mingi Kwon, Youngjung Uh

Global Leadership Division

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Foreground-aware image synthesis aims to generate images as well as their foreground masks. A common approach is to formulate an image as a masked blending of a foreground image and a background image. It is a challenging problem because it is prone to reach the trivial solution where either image overwhelms the other, i.e., the masks become completely full or empty, and the foreground and background are not meaningfully separated. We present FurryGAN with three key components: 1) imposing both the foreground image and the composite image to be realistic, 2) designing a mask as a combination of coarse and fine masks, and 3) guiding the generator by an auxiliary mask predictor in the discriminator. Our method produces realistic images with remarkably detailed alpha masks which cover hair, fur, and whiskers in a fully unsupervised manner. Project page: https://jeongminb.github.io/FurryGAN/.

Original language	English
Title of host publication	Computer Vision – ECCV 2022 - 17th European Conference, Proceedings
Editors	Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, Tal Hassner
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	696-712
Number of pages	17
ISBN (Print)	9783031197802
DOIs	https://doi.org/10.1007/978-3-031-19781-9_40
Publication status	Published - 2022
Event	17th European Conference on Computer Vision, ECCV 2022 - Tel Aviv, Israel Duration: 2022 Oct 23 → 2022 Oct 27

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13674 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	17th European Conference on Computer Vision, ECCV 2022
Country/Territory	Israel
City	Tel Aviv
Period	22/10/23 → 22/10/27

Bibliographical note

Funding Information:
Acknowledgement. This work was supported by the National Research Foundation of Korea(NRF) grant (No. 2022R1F1A107624111) funded by the Korea government (MSIT).

Publisher Copyright:
© 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-031-19781-9_40

Cite this

Bae, J., Kwon, M., & Uh, Y. (2022). FurryGAN: High Quality Foreground-Aware Image Synthesis. In S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, & T. Hassner (Eds.), Computer Vision – ECCV 2022 - 17th European Conference, Proceedings (pp. 696-712). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13674 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-19781-9_40

Bae, Jeongmin ; Kwon, Mingi ; Uh, Youngjung. / FurryGAN : High Quality Foreground-Aware Image Synthesis. Computer Vision – ECCV 2022 - 17th European Conference, Proceedings. editor / Shai Avidan ; Gabriel Brostow ; Moustapha Cissé ; Giovanni Maria Farinella ; Tal Hassner. Springer Science and Business Media Deutschland GmbH, 2022. pp. 696-712 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{b30ec306dca0415586868713197e7f5a,

title = "FurryGAN: High Quality Foreground-Aware Image Synthesis",

abstract = "Foreground-aware image synthesis aims to generate images as well as their foreground masks. A common approach is to formulate an image as a masked blending of a foreground image and a background image. It is a challenging problem because it is prone to reach the trivial solution where either image overwhelms the other, i.e., the masks become completely full or empty, and the foreground and background are not meaningfully separated. We present FurryGAN with three key components: 1) imposing both the foreground image and the composite image to be realistic, 2) designing a mask as a combination of coarse and fine masks, and 3) guiding the generator by an auxiliary mask predictor in the discriminator. Our method produces realistic images with remarkably detailed alpha masks which cover hair, fur, and whiskers in a fully unsupervised manner. Project page: https://jeongminb.github.io/FurryGAN/.",

author = "Jeongmin Bae and Mingi Kwon and Youngjung Uh",

note = "Funding Information: Acknowledgement. This work was supported by the National Research Foundation of Korea(NRF) grant (No. 2022R1F1A107624111) funded by the Korea government (MSIT). Publisher Copyright: {\textcopyright} 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 17th European Conference on Computer Vision, ECCV 2022 ; Conference date: 23-10-2022 Through 27-10-2022",

year = "2022",

doi = "10.1007/978-3-031-19781-9_40",

language = "English",

isbn = "9783031197802",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "696--712",

editor = "Shai Avidan and Gabriel Brostow and Moustapha Ciss{\'e} and Farinella, {Giovanni Maria} and Tal Hassner",

booktitle = "Computer Vision – ECCV 2022 - 17th European Conference, Proceedings",

address = "Germany",

}

Bae, J, Kwon, M & Uh, Y 2022, FurryGAN: High Quality Foreground-Aware Image Synthesis. in S Avidan, G Brostow, M Cissé, GM Farinella & T Hassner (eds), Computer Vision – ECCV 2022 - 17th European Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13674 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 696-712, 17th European Conference on Computer Vision, ECCV 2022, Tel Aviv, Israel, 22/10/23. https://doi.org/10.1007/978-3-031-19781-9_40

FurryGAN : High Quality Foreground-Aware Image Synthesis. / Bae, Jeongmin; Kwon, Mingi; Uh, Youngjung.

Computer Vision – ECCV 2022 - 17th European Conference, Proceedings. ed. / Shai Avidan; Gabriel Brostow; Moustapha Cissé; Giovanni Maria Farinella; Tal Hassner. Springer Science and Business Media Deutschland GmbH, 2022. p. 696-712 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13674 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - FurryGAN

T2 - 17th European Conference on Computer Vision, ECCV 2022

AU - Bae, Jeongmin

AU - Kwon, Mingi

AU - Uh, Youngjung

N1 - Funding Information: Acknowledgement. This work was supported by the National Research Foundation of Korea(NRF) grant (No. 2022R1F1A107624111) funded by the Korea government (MSIT). Publisher Copyright: © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

PY - 2022

Y1 - 2022

N2 - Foreground-aware image synthesis aims to generate images as well as their foreground masks. A common approach is to formulate an image as a masked blending of a foreground image and a background image. It is a challenging problem because it is prone to reach the trivial solution where either image overwhelms the other, i.e., the masks become completely full or empty, and the foreground and background are not meaningfully separated. We present FurryGAN with three key components: 1) imposing both the foreground image and the composite image to be realistic, 2) designing a mask as a combination of coarse and fine masks, and 3) guiding the generator by an auxiliary mask predictor in the discriminator. Our method produces realistic images with remarkably detailed alpha masks which cover hair, fur, and whiskers in a fully unsupervised manner. Project page: https://jeongminb.github.io/FurryGAN/.

AB - Foreground-aware image synthesis aims to generate images as well as their foreground masks. A common approach is to formulate an image as a masked blending of a foreground image and a background image. It is a challenging problem because it is prone to reach the trivial solution where either image overwhelms the other, i.e., the masks become completely full or empty, and the foreground and background are not meaningfully separated. We present FurryGAN with three key components: 1) imposing both the foreground image and the composite image to be realistic, 2) designing a mask as a combination of coarse and fine masks, and 3) guiding the generator by an auxiliary mask predictor in the discriminator. Our method produces realistic images with remarkably detailed alpha masks which cover hair, fur, and whiskers in a fully unsupervised manner. Project page: https://jeongminb.github.io/FurryGAN/.

UR - http://www.scopus.com/inward/record.url?scp=85142695347&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85142695347&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-19781-9_40

DO - 10.1007/978-3-031-19781-9_40

M3 - Conference contribution

AN - SCOPUS:85142695347

SN - 9783031197802

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 696

EP - 712

BT - Computer Vision – ECCV 2022 - 17th European Conference, Proceedings

A2 - Avidan, Shai

A2 - Brostow, Gabriel

A2 - Cissé, Moustapha

A2 - Farinella, Giovanni Maria

A2 - Hassner, Tal

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 23 October 2022 through 27 October 2022

ER -

Bae J, Kwon M, Uh Y. FurryGAN: High Quality Foreground-Aware Image Synthesis. In Avidan S, Brostow G, Cissé M, Farinella GM, Hassner T, editors, Computer Vision – ECCV 2022 - 17th European Conference, Proceedings. Springer Science and Business Media Deutschland GmbH. 2022. p. 696-712. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-19781-9_40

FurryGAN: High Quality Foreground-Aware Image Synthesis

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this