7 citations · 9 across the 4 of their papers we have counts for
7 papers
Tracing Subliminal Learning Through Pretraining
Angela Tang, Vishesh Anand, Gonçalo Paulo
Large language models trained on data generated by other models can sometimes inherit traits unrelated to that data, a phenomenon called subliminal learning. We study both hard lab…
The Unequal Influence of Bad Advice: Using Training Data Attribution to Modulate Emergent Misalignment
Gonçalo Paulo, Louis Jaburi, Nora Belrose +2
Fine-tuning large language models on narrow, misaligned tasks can undo their post-training alignment and induce novel misaligned behaviors -- a phenomenon known as \emph{emergent m…
Can Data Attribution Filter Out Subliminal Learning? Not Reliably
Moritz Weckbecker, Sweta Jena, Jonas Müller +5
Subliminal learning allows language models to transmit behavioral traits through training data with no obvious semantic relationship to those traits, undermining content-based data…
Bergson: An Open Source Library for Data Attribution
Lucia Quirke, Louis Jaburi, David Johnston +6
Data attribution is a promising field in interpretability that aims to explain model behavior through the influence of its training data, with applications including debugging unde…
Concept Influence: Leveraging Interpretability to Improve Performance and Efficiency in Training Data Attribution
Matthew Kowal, Goncalo Paulo, Louis Jaburi +6
As large language models are increasingly trained and fine-tuned, practitioners need methods to identify which training data drive specific behaviors, particularly unintended ones.…
Partially Rewriting a Transformer in Natural Language
Gonçalo Paulo, Nora Belrose
The greatest ambition of mechanistic interpretability is to completely rewrite deep neural networks in a format that is more amenable to human understanding, while preserving their…
Does Transformer Interpretability Transfer to RNNs?
Gonçalo Paulo, Thomas Marshall, Nora Belrose
Recent advances in recurrent neural network architectures, such as Mamba and RWKV, have enabled RNNs to match or exceed the performance of equal-size transformers in terms of langu…