4 papers · 1 filter
Copying Before Suppression: What Drives a Below-Chance Dip During Language Model Training?
Tejas Dahiya, Cole Blondin
Mechanistic interpretability usually studies fully trained models, yet the computations that drive a behaviour can change while the model is still learning the task. On the Indirec…
Interpreting the Latent Structure of Operator Precedence in Language Models
Dharunish Yugeswardeenoo, Harshil Nukala, Ved Shah +4
Large Language Models (LLMs) have demonstrated impressive reasoning capabilities but continue to struggle with arithmetic tasks. Prior works largely focus on outputs or prompting s…
Semantic Convergence: Investigating Shared Representations Across Scaled LLMs
Daniel Son, Sanjana Rathore, Andrew Rufail +6
We investigate feature universality in Gemma-2 language models (Gemma-2-2B and Gemma-2-9B), asking whether models with a four-fold difference in scale still converge on comparable…
Causal Language Control in Multilingual Transformers via Sparse Feature Steering
Cheng-Ting Chou, George Liu, Jessica Sun +4
Deterministically controlling the target generation language of large multilingual language models (LLMs) remains a fundamental challenge, particularly in zero-shot settings where…