Defense via Consensus: Mitigating Subliminal Learning through Multi-Teacher Distillation

Adhyyan Narang, Artin Tajdini, Claire Zhang +1

Fine-tuning can introduce unwanted behaviors through deliberate poisoning or unexpected generalization from seemingly benign data. Because these effects may be difficult to anticip…