On Mitigation of Subliminal Learning in Large Language Models

Atsushi Yanagisawa, Brendan Gho, Rajendran Ramesh Babu Manoj Narender +3

Knowledge distillation can transmit unintended behavioral traits from a teacher model to a student through training data that appear semantically unrelated to those traits, a pheno…