Abstract
We introduce a new problem of generating an image based on a small number of key local patches without any geometric prior. In this work, key local patches are defined as informative regions of the target object or scene. This is a challenging problem since it requires generating realistic images and predicting locations of parts at the same time. We construct adversarial networks to tackle this problem. A generator network generates a fake image as well as a mask based on the encoder-decoder framework. On the other hand, a discriminator network aims to detect fake images. The network is trained with three losses to consider spatial, appearance, and adversarial information. The spatial loss determines whether the locations of predicted parts are correct. Input patches are restored in the output image without much modification due to the appearance loss. The adversarial loss ensures output images are realistic. The proposed network is trained without supervisory signals since no labels of key parts are required. Experimental results on seven datasets demonstrate that the proposed algorithm performs favorably on challenging objects and scenes.
| Original language | English |
|---|---|
| Title of host publication | Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings |
| Editors | Vittorio Ferrari, Cristian Sminchisescu, Martial Hebert, Yair Weiss |
| Publisher | Springer Verlag |
| Pages | 21-37 |
| Number of pages | 17 |
| ISBN (Print) | 9783030012274 |
| DOIs | |
| Publication status | Published - 2018 Jan 1 |
| Event | 15th European Conference on Computer Vision, ECCV 2018 - Munich, Germany Duration: 2018 Sep 8 → 2018 Sep 14 |
Publication series
| Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
|---|---|
| Volume | 11209 LNCS |
| ISSN (Print) | 0302-9743 |
| ISSN (Electronic) | 1611-3349 |
Other
| Other | 15th European Conference on Computer Vision, ECCV 2018 |
|---|---|
| Country | Germany |
| City | Munich |
| Period | 18/9/8 → 18/9/14 |
Fingerprint
All Science Journal Classification (ASJC) codes
- Theoretical Computer Science
- Computer Science(all)
Cite this
}
Unsupervised Holistic Image Generation from Key Local Patches. / Lee, Donghoon; Yun, Sangdoo; Choi, Sungjoon; Yoo, Hwiyeon; Yang, Ming Hsuan; Oh, Songhwai.
Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. ed. / Vittorio Ferrari; Cristian Sminchisescu; Martial Hebert; Yair Weiss. Springer Verlag, 2018. p. 21-37 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11209 LNCS).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
TY - GEN
T1 - Unsupervised Holistic Image Generation from Key Local Patches
AU - Lee, Donghoon
AU - Yun, Sangdoo
AU - Choi, Sungjoon
AU - Yoo, Hwiyeon
AU - Yang, Ming Hsuan
AU - Oh, Songhwai
PY - 2018/1/1
Y1 - 2018/1/1
N2 - We introduce a new problem of generating an image based on a small number of key local patches without any geometric prior. In this work, key local patches are defined as informative regions of the target object or scene. This is a challenging problem since it requires generating realistic images and predicting locations of parts at the same time. We construct adversarial networks to tackle this problem. A generator network generates a fake image as well as a mask based on the encoder-decoder framework. On the other hand, a discriminator network aims to detect fake images. The network is trained with three losses to consider spatial, appearance, and adversarial information. The spatial loss determines whether the locations of predicted parts are correct. Input patches are restored in the output image without much modification due to the appearance loss. The adversarial loss ensures output images are realistic. The proposed network is trained without supervisory signals since no labels of key parts are required. Experimental results on seven datasets demonstrate that the proposed algorithm performs favorably on challenging objects and scenes.
AB - We introduce a new problem of generating an image based on a small number of key local patches without any geometric prior. In this work, key local patches are defined as informative regions of the target object or scene. This is a challenging problem since it requires generating realistic images and predicting locations of parts at the same time. We construct adversarial networks to tackle this problem. A generator network generates a fake image as well as a mask based on the encoder-decoder framework. On the other hand, a discriminator network aims to detect fake images. The network is trained with three losses to consider spatial, appearance, and adversarial information. The spatial loss determines whether the locations of predicted parts are correct. Input patches are restored in the output image without much modification due to the appearance loss. The adversarial loss ensures output images are realistic. The proposed network is trained without supervisory signals since no labels of key parts are required. Experimental results on seven datasets demonstrate that the proposed algorithm performs favorably on challenging objects and scenes.
UR - http://www.scopus.com/inward/record.url?scp=85055084000&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85055084000&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-01228-1_2
DO - 10.1007/978-3-030-01228-1_2
M3 - Conference contribution
AN - SCOPUS:85055084000
SN - 9783030012274
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 21
EP - 37
BT - Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings
A2 - Ferrari, Vittorio
A2 - Sminchisescu, Cristian
A2 - Hebert, Martial
A2 - Weiss, Yair
PB - Springer Verlag
ER -