PCAE: A Framework of Plug-in Conditional Auto-Encoder for Controllable Text Generation
arXiv:2210.03496 · doi:10.1016/j.knosys.2022.109766
Abstract
Controllable text generation has taken a gigantic step forward these days. Yet existing methods are either constrained in a one-off pattern or not efficient enough for receiving multiple conditions at every generation stage. We propose a model-agnostic framework Plug-in Conditional Auto-Encoder for Controllable Text Generation (PCAE) towards flexible and semi-supervised text generation. Our framework is "plug-and-play" with partial parameters to be fine-tuned in the pre-trained model (less than a half). Crucial to the success of PCAE is the proposed broadcasting label fusion network for navigating the global latent code to a specified local and confined space. Visualization of the local latent prior well confirms the primary devotion in hidden space of the proposed model. Moreover, extensive experiments across five related generation tasks (from 2 conditions up to 10 conditions) on both RNN- based and pre-trained BART [26] based auto-encoders reveal the high capability of PCAE, which enables generation that is highly manipulable, syntactically diverse and time-saving with minimum labeled samples. We will release our code at https://github.com/ImKeTT/pcae.
arXiv commentKnowledge-Based Systems
References in corpus (7)
- Adam: A Method for Stochastic Optimization
- CTRL: A Conditional Transformer Language Model for Controllable Generation
- Style Transfer from Non-Parallel Text by Cross-Alignment
- InfoVAE: Information Maximizing Variational Autoencoders
- Transformer-based Conditional Variational Autoencoder for Controllable Story Generation
- Neural Language Generation: Formulation, Methods, and Evaluation
- AdaVAE: Exploring Adaptive GPT-2s in Variational Auto-Encoders for Language Modeling