Reducing Non-Normative Text Generation from Language Models

Xiangyu Peng, Siyan Li, Spencer Frazier +1

Large-scale, transformer-based language models such as GPT-2 are pretrained on diverse corpora scraped from the internet. Consequently, they are prone to generating non-normative t…