Winter Soldier: Backdooring Language Models at Pre-Training with Indirect Data Poisoning

Wassim Bouaziz, Mathurin Videau, Nicolas Usunier +1

The pre-training of large language models (LLMs) relies on massive text datasets sourced from diverse and difficult-to-curate origins. Although membership inference attacks and hid…