Subliminal Learning is Non-Semantic Distillation

Ethan Hadley, Eren Gultepe

Subliminal Learning (SL) is a surprising type of generalization displayed by modern language models. It allows the transfer of a bias or behavior from a teacher model to a student…