11 papers
When Forgetting is not Catastrophic: On the Mechanics of Spurious Forgetting
Vedant Palit, Florent Draye, Nicolas Zucchet +2
Knowledge that a language model appears to forget during finetuning often remains stored and can be recovered, a phenomenon called spurious forgetting. Finetuning on new facts can…
ABC: Advantage-Based Control Variates for Reinforcement Learning with Verifiable Rewards
Hsiao-Ru Pan, Florent Draye, Bernhard Schölkopf
Recent progress in reinforcement learning with verifiable rewards (RLVR) has highlighted the effectiveness of simple critic-free policy-gradient methods such as Group Relative Poli…
RAPTOR: Role-Aware Private Training for Mixture-of-Experts
Duc Dm, Khai Le-Duc, Nguyen Do +18
Differentially private (DP) fine-tuning methods treat sparse Mixture-of-Experts (MoE) models as a single dense block, ignoring that shared layers see all data while experts only se…
How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs?
Prakhar Gupta, Terry Jingchen Zhang, Florent Draye +2
Modern LLMs are alarmingly susceptible to surprisingly simple immaterial changes of input prompts: a casual hint, an incorrectly labeled few-shot example, or a fake prior assistant…
How Do Linear Probes Emerge? A Circuit-Tracing Framework with Concept-Targeted Attribution
Vedant Palit, Florent Draye, Terry Jingchen Zhang +2
Transcoder attribution graphs are usually trained to explain why a model assigns high probability to a particular next token. We introduce Concept-Targeted Attribution (CTA), which…
STRIDE: Training Data Attribution via Sparse Recovery from Subset Perturbations
Rishit Dagli, Abir Harrasse, Luke Zhang +4
Training Data Attribution (TDA) seeks to trace a model's predictions back to its training data. The gold standard for TDA relies on causal interventions, observing how a model chan…
PruneGround: Plug-and-play Spatial Pruning for 3D Visual Grounding
Duc Cao Dinh, Khai Le-Duc, Florent Draye +4
3D Visual Grounding (3DVG) aims to localize target objects in 3D scenes given natural language descriptions. Existing approaches typically perform reasoning over the entire scene,…
CLT-Forge: A Scalable Library for Cross-Layer Transcoders and Attribution Graphs
Florent Draye, Vedant Palit, Abir Harrasse +8
Mechanistic interpretability seeks to understand how Large Language Models (LLMs) represent and process information. Recent approaches based on dictionary learning and transcoders…
Superposition Without Interference? Towards Isolated Interventions via Almost Orthogonal Features in Language Models
Moritz Miller, Florent Draye, Bernhard Schölkopf
A central premise in mechanistic interpretability is that meaningful concepts in language models are represented by linear features in activation space. For such features to suppor…
Intrinsically Interpretable Attention via Sparse Post-Training
Florent Draye, Anson Lei, Hsiao-Ru Pan +2
We introduce a simple post-training method that makes transformer attention sparse without sacrificing performance. Applying a flexible sparsity regularisation under a constrained-…
Tracing Multilingual Representations in LLMs with Cross-Layer Transcoders
Abir Harrasse, Florent Draye, Punya Syon Pandey +2
Multilingual Large Language Models (LLMs) can process many languages, yet how they internally represent this diversity remains unclear. Do they form shared multilingual representat…