TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling
arXiv:2010.03802
Abstract
We present a novel approach to the problem of text style transfer. Unlike previous approaches requiring style-labeled training data, our method makes use of readily-available unlabeled text by relying on the implicit connection in style between adjacent sentences, and uses labeled data only at inference time. We adapt T5 (Raffel et al., 2020), a strong pretrained text-to-text model, to extract a style vector from text and use it to condition the decoder to perform style transfer. As our label-free training results in a style vector space encoding many facets of style, we recast transfers as "targeted restyling" vector operations that adjust specific attributes of the input while preserving others. We demonstrate that training on unlabeled Amazon reviews data results in a model that is competitive on sentiment transfer, even compared to models trained fully on labeled data. Furthermore, applying our novel method to a diverse corpus of unlabeled web text results in a single model capable of transferring along multiple dimensions of style (dialect, emotiveness, formality, politeness, sentiment) despite no additional training and using only a handful of exemplars at inference time.
References in corpus (9)
- CTRL: A Conditional Transformer Language Model for Controllable Generation
- Style Transfer from Non-Parallel Text by Cross-Alignment
- Multi-Task Neural Models for Translating Between Styles Within and Across Languages
- A Probabilistic Formulation of Unsupervised Text Style Transfer
- Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach
- Unsupervised Text Style Transfer using Language Models as Discriminators
- Evaluating prose style transfer with the Bible
- On Variational Learning of Controllable Representations for Text without Supervision
- Language Style Transfer from Sentences with Arbitrary Unknown Styles