Diversity regularized autoencoders for text generation

Hyeseon Ko, Junhyuk Lee, Jinhong Kim, Jongwuk Lee, Hyunjung Shim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we propose a simple yet powerful text generation model, called diversity regularized autoencoders (DRAE). The key novelty of the proposed model lies in its ability to handle various sentence modifications such as insertions, deletions, substitutions, and maskings, and to take them as input. Because the noise-injection strategy enables an encoder to make the latent distribution smooth and continuous, the proposed model can generate more diverse and coherent sentences. Also, we adopt the Wasserstein generative adversarial networks with a gradient penalty to achieve stable adversarial training of the prior distribution. We evaluate the proposed model using quantitative, qualitative, and human evaluations on two public datasets. Experimental results demonstrate that our model using a noise-injection strategy produces more natural and diverse sentences than several baseline models. Furthermore, it is found that our model shows the synergistic effect of grammar correction and paraphrase generation in an unsupervised way.

Original languageEnglish
Title of host publication35th Annual ACM Symposium on Applied Computing, SAC 2020
PublisherAssociation for Computing Machinery
Pages883-891
Number of pages9
ISBN (Electronic)9781450368667
DOIs
Publication statusPublished - 2020 Mar 30
Event35th Annual ACM Symposium on Applied Computing, SAC 2020 - Brno, Czech Republic
Duration: 2020 Mar 302020 Apr 3

Publication series

NameProceedings of the ACM Symposium on Applied Computing

Conference

Conference35th Annual ACM Symposium on Applied Computing, SAC 2020
CountryCzech Republic
CityBrno
Period20/3/3020/4/3

All Science Journal Classification (ASJC) codes

  • Software

Fingerprint Dive into the research topics of 'Diversity regularized autoencoders for text generation'. Together they form a unique fingerprint.

  • Cite this

    Ko, H., Lee, J., Kim, J., Lee, J., & Shim, H. (2020). Diversity regularized autoencoders for text generation. In 35th Annual ACM Symposium on Applied Computing, SAC 2020 (pp. 883-891). (Proceedings of the ACM Symposium on Applied Computing). Association for Computing Machinery. https://doi.org/10.1145/3341105.3373998