385 citations · 1.5k across the 36 of their papers we have counts for
7 papers · 1 filter
In-Context Algebra
Eric Todd, Jannik Brinkmann, Rohit Gandikota +1
We investigate the mechanisms that arise when transformers are trained to solve arithmetic on sequences where tokens are variables whose meaning is determined only through their in…
Vector Arithmetic in Concept and Token Subspaces
Sheridan Feucht, Byron Wallace, David Bau
In order to predict the next token, LLMs must represent semantic and surface-level information about the current word. Previous work identified two types of attention heads that di…
LLMs Encode Harmfulness and Refusal Separately
Jiachen Zhao, Jing Huang, Zhengxuan Wu +2
LLMs are trained to refuse harmful instructions, but do they truly understand harmfulness beyond just refusing? Prior work has shown that LLMs' refusal behaviors can be mediated by…
Discovering Forbidden Topics in Language Models
Can Rager, Chris Wendler, Rohit Gandikota +1
Refusal discovery is the task of identifying the full set of topics that a language model refuses to discuss. We introduce this new problem setting and develop a refusal discovery…
When Are Concepts Erased From Diffusion Models?
Kevin Lu, Nicky Kriplani, Rohit Gandikota +4
In concept erasure, a model is modified to selectively prevent it from generating a target concept. Despite the rapid development of new methods, it remains unclear how thoroughly…
Distilling Diversity and Control in Diffusion Models
Rohit Gandikota, David Bau
Distilled diffusion models generate images in far fewer timesteps but suffer from reduced sample diversity when generating multiple outputs from the same prompt. To understand this…