Papers about controllable text generation (CTG) via latent auto-encoders (AEs). Mainly focus on open-domain sentence generation with some style transfer generation methods (without dialogue generation for now).
Mostly for Variational Auto-Encoders (VAEs)- Reasearchgate (2020, THU) / The Road from MLE to EM to VAE: A Brief Tutorial / TL;DR
- EMNLP (2018, Harvard) / A Tutorial on Deep Latent Variable Models of Natural Language / TL; DR
- Arxiv (2016, Carl Doersch) / Tutorial on Variational Autoencoders / Complete and the first VAE tutorial, last updated on Jan. 2021
- Hard Control: Knowledge/Keyword/Table-Driven controllable generation is denoted as K2T;
- Soft Control: Globally Sentiment / Tense / Topic controllable generation is denoted as G2T.
List format follows:
Publication info. / paper and link / TL; DR / Code link (if available) / Chinese Blog Link (if available)
### 2021- To be continued..
- EMNLP (Tsinghua) / Long and Diverse Text Generation with Planning-based Hierarchical Variational Model / K2T, 2 latent variable models for keywords assignment plan of every sentence and word generation respectively. / Code
- ICASSP (Alibaba) / Improve Diverse Text Generation by Self Labeling Conditional Variational Auto Encoder / K2T,
- NeurIPS (PKU) / Controllable Unsupervised Text Attribute Transfer via Editing Entangled Latent Representation / G2T, style transfer generation
- Arxiv (Buffalo Univ.) / Transformer-based Conditional Variational Autoencoder for Controllable Story Generation / G2T, explored 3 different methods for condition combination with GPT-2 as both encoder and decoder of a text VAE. / Nan
- Arxiv (EPFL) / Bag-of-Vectors Autoencoders For Unsupervised Conditional Text Generation / G2T, style transfer task /
-
ACL (Wuhan Univ.) / Pre-train and Plug-in: Flexible Conditional Text Generation with Variational Auto-Encoders / G2T, the first "Plug-and-Play" latent AE consists of a pretrain VAE and
$n$ plug-in VAE for$n$ given conditions. / Code / Chinese Blog - ACL (Duke) / Improving Disentangled Text Representation Learning with Information-Theoretic Guidance / G2T, explained with variation of information theory. 2 encoders for style and context encoding to produce distinct latents, a discriminator with style label for style latent adversarial learning and a VAE for context learning, concat two latents for controllable generation. / Nan
- EMNLP (EPFL) / Plug and Play Autoencoders for Conditional Text Generation / G2T, style transfer task
- ICLR (ByteDance) / Variational Template Machine For Data-to-Text Generation / K2T, use VAE to generate keyword templates, fill pre-assigned keywords into sampled template. / Code
- To be continued..
- NIPS (Michigan Univ.) / Content preserving text generation with attribute controls / G2T, style transfer task
- ICML (CMU) / Improved Variational Autoencoders for Text Modeling using Dilated Convolutions / G2T, self-supervised and semi-supervised
- Findings (Manchester Univ.) / Disentangling Generative Factors in Natural Language with Discrete Variational Autoencoders / G2T, model every condition into a discrete latent and uses Gumbel softmax for back-prop. Decomposes KL regularization loss into 3 terms related to disentanglement learning like the one described in TC-VAE / Nan
- NeurIPS (UMBC) / A Discrete Variational Recurrent Topic Model without the Reparametrization Trick / G2T, model word-level topic latent codes using continued multiplication approximation, and several auxiliary loss w.r.t. word-level and document-level topic correlation optimization. / Code
- ICML (MIT) / Educating Text Autoencoders: Latent Representation Guidance via Denoising / G2T, add noise at input token level to avoid token-latent irrelevance issue of text latent AEs. / Code
- ICML(ByteDance) / Dispersed Exponential Family Mixture VAEs for Interpretable Text Generation / G2T, mix gaussian model (1 gaussian 1 topic ideally) for VAE prior modeling. / Code
-
ICML (Borealis) / On Variational Learning of Controllable Representations for Text without Supervision / G2T, first identify the latent vacancy issue in text VAE, use GloVe and RNN embedding as two distinct latents (
$z_1,z_2$ ). Imposes orthogonal and reconstructing regularization loss on$z_1$ . / Code / Chinese Blog
- EMNLP (CAS) / A Topic Augmented Text Generation Model: Joint Learning of Semantics and Structural Features / G2T, model text semantic and structural features via 2 separate VAEs, concat the distinct latent codes for controllable generation. / Nan
- NAACL (Duke) / Topic-Guided Variational Autoencoders for Text Generation / G2T, consists of a latent topic model whose latent is a GMM (each Gaussian is a topic ideally) and modeled by Householder Flow, and a sequence VAE that takes the same latent for generation. / Nan
- EMNLP (Buffalo Univ.) / Implicit Deep Latent Variable Models for Text Generation / G2T, add an auxiliary mutual information between observed data and latent variable based on vanilla text VAE in order to educate a more meaningful latent space. / Code
- ACL (Nanjing Univ.) / Generating Sentences from Disentangled Syntactic and Semantic Spaces / G2T,
- Arxiv (Waterloo Univ.) / Stylized Text Generation Using Wasserstein Autoencoders with a Mixture of Gaussian Prior / G2T,
- AISTATS (Duke) / Topic Compositional Neural Language Model / G2T, a VAE to model topic distributions of documents and a muti-expert LSTM network for controllable generation. / Nan
- Arxiv (UCSB) / Dirichlet Variational Autoencoder for Text Modeling / G2T, a plain VAE for sequence modeling ,and a VAE parameterized by Dirichlet for topic modeling whose latent posterior is conditioned on the sequence latent. / Nan