Improving Formality Style Transfer with Context-Aware Rule Injection
arXiv:2106.00210
Abstract
Models pre-trained on large-scale regular text corpora often do not work well for user-generated data where the language styles differ significantly from the mainstream text. Here we present Context-Aware Rule Injection (CARI), an innovative method for formality style transfer (FST). CARI injects multiple rules into an end-to-end BERT-based encoder and decoder model. It learns to select optimal rules based on context. The intrinsic evaluation showed that CARI achieved the new highest performance on the FST benchmark dataset. Our extrinsic evaluation showed that CARI can greatly improve the regular pre-trained models' performance on several tweet sentiment analysis tasks.
arXiv commentACL2021
References in corpus (9)
- Adam: A Method for Stochastic Optimization
- Multi-Task Neural Models for Translating Between Styles Within and Across Languages
- Low-Memory Neural Network Training: A Technical Report
- Formality Style Transfer with Hybrid Textual Annotations
- MoNoise: Modeling Noise Using a Modular Normalization System
- Zero-shot Entity Linking with Efficient Long Range Sequence Modeling
- Generating Sentences from Disentangled Syntactic and Semantic Spaces
- Controllable Paraphrase Generation with a Syntactic Exemplar
- Parallel Data Augmentation for Formality Style Transfer