Tracing Subliminal Learning Through Pretraining

Angela Tang, Vishesh Anand, Gonçalo Paulo

Large language models trained on data generated by other models can sometimes inherit traits unrelated to that data, a phenomenon called subliminal learning. We study both hard lab…