SHAPED: Shared-Private Encoder-Decoder for Text Style Adaptation
arXiv:1804.04093
Abstract
Supervised training of abstractive language generation models results in learning conditional probabilities over language sequences based on the supervised training signal. When the training signal contains a variety of writing styles, such models may end up learning an 'average' style that is directly influenced by the training data make-up and cannot be controlled by the needs of an application. We describe a family of model architectures capable of capturing both generic language characteristics via shared model parameters, as well as particular style characteristics via private model parameters. Such models are able to generate language according to a specific learned style, while still taking advantage of their power to model generic language phenomena. Furthermore, we describe an extension that uses a mixture of output distributions from all learned styles to perform on-the fly style adaptation based on the textual input alone. Experimentally, we find that the proposed models consistently outperform models that encapsulate single-style or average-style language generation capabilities.
arXiv commentNAACL 2018
References in corpus (13)
- Sequence to Sequence Learning with Neural Networks
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Frustratingly Easy Domain Adaptation
- A Deep Reinforced Model for Abstractive Summarization
- Unsupervised Domain Adaptation with Residual Transfer Networks
- Sequence Level Training with Recurrent Neural Networks
- Domain Separation Networks
- Deep Reinforcement Learning for Dialogue Generation
- Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond
- Adversarial Learning for Neural Dialogue Generation
- Sentence Simplification with Deep Reinforcement Learning
- A Pilot Study of Domain Adaptation Effect for Neural Abstractive Summarization