42 citations · 76 across the 13 of their papers we have counts for
10 papers · 1 filter
When Models Manipulate Manifolds: The Geometry of a Counting Task
Wes Gurnee, Emmanuel Ameisen, Isaac Kauvar +4
Language models can perceive visual properties of text despite receiving only sequences of tokens-we mechanistically investigate how Claude 3.5 Haiku accomplishes one such task: li…
The Remarkable Robustness of LLMs: Stages of Inference?
Vedang Lad, Jin Hwa Lee, Wes Gurnee +1
We investigate the robustness of Large Language Models (LLMs) to structural interventions by deleting and swapping adjacent layers during inference. Surprisingly, models retain 72-…
Confidence Regulation Neurons in Language Models
Alessandro Stolfo, Ben Wu, Wes Gurnee +4
Despite their widespread use, the mechanisms by which large language models (LLMs) represent and regulate uncertainty in next-token predictions remain largely unexplored. This stud…
Refusal in Language Models Is Mediated by a Single Direction
Andy Arditi, Oscar Obeso, Aaquib Syed +4
Conversational large language models are fine-tuned for both instruction-following and safety, resulting in models that obey benign requests but refuse harmful ones. While this ref…
Not All Language Model Features Are One-Dimensionally Linear
Joshua Engels, Eric J. Michaud, Isaac Liao +2
Recent work has proposed that language models perform computation by manipulating one-dimensional representations of concepts ("features") in activation space. In contrast, we expl…
Universal Neurons in GPT2 Language Models
Wes Gurnee, Theo Horsley, Zifan Carl Guo +5
A basic question within the emerging field of mechanistic interpretability is the degree to which neural networks learn the same underlying mechanisms. In other words, are neural m…
Training Dynamics of Contextual N-Grams in Language Models
Lucia Quirke, Lovis Heindrich, Wes Gurnee +1
Prior work has shown the existence of contextual neurons in language models, including a neuron that activates on German text. We show that this neuron exists within a broader cont…
Language Models Represent Space and Time
Wes Gurnee, Max Tegmark
The capabilities of large language models (LLMs) have sparked debate over whether such systems just learn an enormous collection of superficial statistics or a set of more coherent…
Finding Neurons in a Haystack: Case Studies with Sparse Probing
Wes Gurnee, Neel Nanda, Matthew Pauly +3
Despite rapid adoption and deployment of large language models (LLMs), the internal computations of these models remain opaque and poorly understood. In this work, we seek to under…
Learning Sparse Nonlinear Dynamics via Mixed-Integer Optimization
Dimitris Bertsimas, Wes Gurnee
Discovering governing equations of complex dynamical systems directly from data is a central problem in scientific machine learning. In recent years, the sparse identification of n…