42 citations · 76 across the 13 of their papers we have counts for
14 papers
A Shared Subcircuit Lets LLMs Count Down Across Tasks
Jacob Dunefsky, Wes Gurnee, Emmanuel Ameisen
Writing a sentence of exactly twelve words; ending a DNA sequence at the right codon; formatting an ASCII table. These are all tasks that language models can do that requires track…
When Models Manipulate Manifolds: The Geometry of a Counting Task
Wes Gurnee, Emmanuel Ameisen, Isaac Kauvar +4
Language models can perceive visual properties of text despite receiving only sequences of tokens-we mechanistically investigate how Claude 3.5 Haiku accomplishes one such task: li…
Multilevel Interpretability Of Artificial Neural Networks: Leveraging Framework And Methods From Neuroscience
Zhonghao He, Jascha Achterberg, Katie Collins +13
As deep learning systems are scaled up to many billions of parameters, relating their internal structure to external behaviors becomes very challenging. Although daunting, this pro…
The Remarkable Robustness of LLMs: Stages of Inference?
Vedang Lad, Jin Hwa Lee, Wes Gurnee +1
We investigate the robustness of Large Language Models (LLMs) to structural interventions by deleting and swapping adjacent layers during inference. Surprisingly, models retain 72-…
Confidence Regulation Neurons in Language Models
Alessandro Stolfo, Ben Wu, Wes Gurnee +4
Despite their widespread use, the mechanisms by which large language models (LLMs) represent and regulate uncertainty in next-token predictions remain largely unexplored. This stud…
Refusal in Language Models Is Mediated by a Single Direction
Andy Arditi, Oscar Obeso, Aaquib Syed +4
Conversational large language models are fine-tuned for both instruction-following and safety, resulting in models that obey benign requests but refuse harmful ones. While this ref…
Not All Language Model Features Are One-Dimensionally Linear
Joshua Engels, Eric J. Michaud, Isaac Liao +2
Recent work has proposed that language models perform computation by manipulating one-dimensional representations of concepts ("features") in activation space. In contrast, we expl…
Universal Neurons in GPT2 Language Models
Wes Gurnee, Theo Horsley, Zifan Carl Guo +5
A basic question within the emerging field of mechanistic interpretability is the degree to which neural networks learn the same underlying mechanisms. In other words, are neural m…
Training Dynamics of Contextual N-Grams in Language Models
Lucia Quirke, Lovis Heindrich, Wes Gurnee +1
Prior work has shown the existence of contextual neurons in language models, including a neuron that activates on German text. We show that this neuron exists within a broader cont…
Language Models Represent Space and Time
Wes Gurnee, Max Tegmark
The capabilities of large language models (LLMs) have sparked debate over whether such systems just learn an enormous collection of superficial statistics or a set of more coherent…
Finding Neurons in a Haystack: Case Studies with Sparse Probing
Wes Gurnee, Neel Nanda, Matthew Pauly +3
Despite rapid adoption and deployment of large language models (LLMs), the internal computations of these models remain opaque and poorly understood. In this work, we seek to under…
Learning Sparse Nonlinear Dynamics via Mixed-Integer Optimization
Dimitris Bertsimas, Wes Gurnee
Discovering governing equations of complex dynamical systems directly from data is a central problem in scientific machine learning. In recent years, the sparse identification of n…
Combatting Gerrymandering with Ranked Choice Voting: An Experimental Analysis of Multi-member Districts in the United States
Nikhil Garg, Wes Gurnee, David Rothschild +1
Every representative democracy must specify a mechanism under which voters choose their representatives. The most common mechanism in the United States -- Winner takes all single-m…
Fairmandering: A column generation heuristic for fairness-optimized political districting
Wes Gurnee, David B. Shmoys
The American winner-take-all congressional district system empowers politicians to engineer electoral outcomes by manipulating district boundaries. Existing computational solutions…