Emotional Landscape Image Generation Using Generative Adversarial Networks

Chanjong Park; In Kwon Lee

doi:10.1007/978-3-030-69538-5_35

Emotional Landscape Image Generation Using Generative Adversarial Networks

Chanjong Park, In Kwon Lee

College of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We design a deep learning framework that generates landscape images that match a given emotion. We are working on a more challenging approach to generate landscape scenes that do not have main objects making it easier to recognize the emotion. To solve this problem, deep networks based on generative adversarial networks are proposed. A new residual unit called emotional residual unit (ERU) is proposed to better reflect the emotion on training. An affective feature matching loss (AFM-loss) optimized for the emotional image generation is also proposed. This approach produced better images according to the given emotions. To demonstrate performance of the proposed model, a set of experiments including user studies was conducted. The results reveal a higher preference in the new model than the previous ones, demonstrating the production of images suitable for the given emotions. Ablation studies demonstrate that the ERU and AFM-loss enhanced the performance of the model.

Original language	English
Title of host publication	Computer Vision – ACCV 2020 - 15th Asian Conference on Computer Vision, 2020, Revised Selected Papers
Editors	Hiroshi Ishikawa, Cheng-Lin Liu, Tomas Pajdla, Jianbo Shi
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	573-590
Number of pages	18
ISBN (Print)	9783030695378
DOIs	https://doi.org/10.1007/978-3-030-69538-5_35
Publication status	Published - 2021
Event	15th Asian Conference on Computer Vision, ACCV 2020 - Virtual, Online Duration: 2020 Nov 30 → 2020 Dec 4

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12625 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	15th Asian Conference on Computer Vision, ACCV 2020
City	Virtual, Online
Period	20/11/30 → 20/12/4

Bibliographical note

Funding Information:
Acknowledgement. This research was supported by the MSIT(Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2020-2018-0-01419) supervised by the IITP (Institute for Information and Communications Technology Planning and Evaluation) and the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2020R1A2C2014622).

Publisher Copyright:
© 2021, Springer Nature Switzerland AG.

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-030-69538-5_35

Cite this

Park, C., & Lee, I. K. (2021). Emotional Landscape Image Generation Using Generative Adversarial Networks. In H. Ishikawa, C-L. Liu, T. Pajdla, & J. Shi (Eds.), Computer Vision – ACCV 2020 - 15th Asian Conference on Computer Vision, 2020, Revised Selected Papers (pp. 573-590). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12625 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-69538-5_35

Park, Chanjong ; Lee, In Kwon. / Emotional Landscape Image Generation Using Generative Adversarial Networks. Computer Vision – ACCV 2020 - 15th Asian Conference on Computer Vision, 2020, Revised Selected Papers. editor / Hiroshi Ishikawa ; Cheng-Lin Liu ; Tomas Pajdla ; Jianbo Shi. Springer Science and Business Media Deutschland GmbH, 2021. pp. 573-590 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{f937987739fe4ecaa41b455e7a975004,

title = "Emotional Landscape Image Generation Using Generative Adversarial Networks",

abstract = "We design a deep learning framework that generates landscape images that match a given emotion. We are working on a more challenging approach to generate landscape scenes that do not have main objects making it easier to recognize the emotion. To solve this problem, deep networks based on generative adversarial networks are proposed. A new residual unit called emotional residual unit (ERU) is proposed to better reflect the emotion on training. An affective feature matching loss (AFM-loss) optimized for the emotional image generation is also proposed. This approach produced better images according to the given emotions. To demonstrate performance of the proposed model, a set of experiments including user studies was conducted. The results reveal a higher preference in the new model than the previous ones, demonstrating the production of images suitable for the given emotions. Ablation studies demonstrate that the ERU and AFM-loss enhanced the performance of the model.",

author = "Chanjong Park and Lee, {In Kwon}",

note = "Funding Information: Acknowledgement. This research was supported by the MSIT(Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2020-2018-0-01419) supervised by the IITP (Institute for Information and Communications Technology Planning and Evaluation) and the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2020R1A2C2014622). Publisher Copyright: {\textcopyright} 2021, Springer Nature Switzerland AG.; 15th Asian Conference on Computer Vision, ACCV 2020 ; Conference date: 30-11-2020 Through 04-12-2020",

year = "2021",

doi = "10.1007/978-3-030-69538-5_35",

language = "English",

isbn = "9783030695378",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "573--590",

editor = "Hiroshi Ishikawa and Cheng-Lin Liu and Tomas Pajdla and Jianbo Shi",

booktitle = "Computer Vision – ACCV 2020 - 15th Asian Conference on Computer Vision, 2020, Revised Selected Papers",

address = "Germany",

}

Park, C & Lee, IK 2021, Emotional Landscape Image Generation Using Generative Adversarial Networks. in H Ishikawa, C-L Liu, T Pajdla & J Shi (eds), Computer Vision – ACCV 2020 - 15th Asian Conference on Computer Vision, 2020, Revised Selected Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12625 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 573-590, 15th Asian Conference on Computer Vision, ACCV 2020, Virtual, Online, 20/11/30. https://doi.org/10.1007/978-3-030-69538-5_35

Emotional Landscape Image Generation Using Generative Adversarial Networks. / Park, Chanjong; Lee, In Kwon.

Computer Vision – ACCV 2020 - 15th Asian Conference on Computer Vision, 2020, Revised Selected Papers. ed. / Hiroshi Ishikawa; Cheng-Lin Liu; Tomas Pajdla; Jianbo Shi. Springer Science and Business Media Deutschland GmbH, 2021. p. 573-590 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12625 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Emotional Landscape Image Generation Using Generative Adversarial Networks

AU - Park, Chanjong

AU - Lee, In Kwon

N1 - Funding Information: Acknowledgement. This research was supported by the MSIT(Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2020-2018-0-01419) supervised by the IITP (Institute for Information and Communications Technology Planning and Evaluation) and the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2020R1A2C2014622). Publisher Copyright: © 2021, Springer Nature Switzerland AG.

PY - 2021

Y1 - 2021

N2 - We design a deep learning framework that generates landscape images that match a given emotion. We are working on a more challenging approach to generate landscape scenes that do not have main objects making it easier to recognize the emotion. To solve this problem, deep networks based on generative adversarial networks are proposed. A new residual unit called emotional residual unit (ERU) is proposed to better reflect the emotion on training. An affective feature matching loss (AFM-loss) optimized for the emotional image generation is also proposed. This approach produced better images according to the given emotions. To demonstrate performance of the proposed model, a set of experiments including user studies was conducted. The results reveal a higher preference in the new model than the previous ones, demonstrating the production of images suitable for the given emotions. Ablation studies demonstrate that the ERU and AFM-loss enhanced the performance of the model.

AB - We design a deep learning framework that generates landscape images that match a given emotion. We are working on a more challenging approach to generate landscape scenes that do not have main objects making it easier to recognize the emotion. To solve this problem, deep networks based on generative adversarial networks are proposed. A new residual unit called emotional residual unit (ERU) is proposed to better reflect the emotion on training. An affective feature matching loss (AFM-loss) optimized for the emotional image generation is also proposed. This approach produced better images according to the given emotions. To demonstrate performance of the proposed model, a set of experiments including user studies was conducted. The results reveal a higher preference in the new model than the previous ones, demonstrating the production of images suitable for the given emotions. Ablation studies demonstrate that the ERU and AFM-loss enhanced the performance of the model.

UR - http://www.scopus.com/inward/record.url?scp=85103255579&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85103255579&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-69538-5_35

DO - 10.1007/978-3-030-69538-5_35

M3 - Conference contribution

AN - SCOPUS:85103255579

SN - 9783030695378

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 573

EP - 590

BT - Computer Vision – ACCV 2020 - 15th Asian Conference on Computer Vision, 2020, Revised Selected Papers

A2 - Ishikawa, Hiroshi

A2 - Liu, Cheng-Lin

A2 - Pajdla, Tomas

A2 - Shi, Jianbo

PB - Springer Science and Business Media Deutschland GmbH

T2 - 15th Asian Conference on Computer Vision, ACCV 2020

Y2 - 30 November 2020 through 4 December 2020

ER -

Park C, Lee IK. Emotional Landscape Image Generation Using Generative Adversarial Networks. In Ishikawa H, Liu C-L, Pajdla T, Shi J, editors, Computer Vision – ACCV 2020 - 15th Asian Conference on Computer Vision, 2020, Revised Selected Papers. Springer Science and Business Media Deutschland GmbH. 2021. p. 573-590. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-69538-5_35

Emotional Landscape Image Generation Using Generative Adversarial Networks

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this