DRIT++: Diverse Image-to-Image Translation via Disentangled Representations

Hsin Ying Lee, Hung Yu Tseng, Qi Mao, Jia Bin Huang, Yu Ding Lu, Maneesh Singh, Ming Hsuan Yang

Research output: Contribution to journalArticle

Abstract

Image-to-image translation aims to learn the mapping between two visual domains. There are two main challenges for this task: (1) lack of aligned training pairs and (2) multiple possible outputs from a single input image. In this work, we present an approach based on disentangled representation for generating diverse outputs without paired training images. To synthesize diverse outputs, we propose to embed images onto two spaces: a domain-invariant content space capturing shared information across domains and a domain-specific attribute space. Our model takes the encoded content features extracted from a given input and attribute vectors sampled from the attribute space to synthesize diverse outputs at test time. To handle unpaired training data, we introduce a cross-cycle consistency loss based on disentangled representations. Qualitative results show that our model can generate diverse and realistic images on a wide range of tasks without paired training data. For quantitative evaluations, we measure realism with user study and Fréchet inception distance, and measure diversity with the perceptual distance metric, Jensen–Shannon divergence, and number of statistically-different bins.

Original languageEnglish
Pages (from-to)2402-2417
Number of pages16
JournalInternational Journal of Computer Vision
Volume128
Issue number10-11
DOIs
Publication statusAccepted/In press - 2020

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'DRIT++: Diverse Image-to-Image Translation via Disentangled Representations'. Together they form a unique fingerprint.

  • Cite this

    Lee, H. Y., Tseng, H. Y., Mao, Q., Huang, J. B., Lu, Y. D., Singh, M., & Yang, M. H. (Accepted/In press). DRIT++: Diverse Image-to-Image Translation via Disentangled Representations. International Journal of Computer Vision, 128(10-11), 2402-2417. https://doi.org/10.1007/s11263-019-01284-z