How Does Controllability Emerge In Language Models During Pretraining?

Jianshu She, Xinyue Li, Eric Xing +2

Language models can be steered by modifying their internal representations to control concepts such as emotion, style, or truthfulness in generation. However, the conditions for an…