84 citations · 226 across the 30 of their papers we have counts for
34 papers
PRISM Edit: One Vector for All Temporal Answers
Chen Huang, Qi Zheng, Ruiqin Zheng +2
Model editing keeps large language models (LLMs) up to date without retraining, but temporal facts expose a limitation of the prevailing locate-and-edit paradigm: an update is not…
HyperSteer: Activation Steering at Scale with Hypernetworks
Jiuding Sun, Sidharth Baskaran, Zhengxuan Wu +3
Steering language models (LMs) by modifying internal activations is a popular approach for controlling text generation. Unsupervised dictionary learning methods, e.g., sparse autoe…
Improved Representation Steering for Language Models
Zhengxuan Wu, Qinan Yu, Aryaman Arora +2
Steering methods for language models (LMs) seek to provide fine-grained and interpretable control over model generations by variously changing model inputs, weights, or representat…
AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Zhengxuan Wu, Aryaman Arora, Atticus Geiger +5
Fine-grained steering of language model outputs is essential for safety and reliability. Prompting and finetuning are widely used to achieve these goals, but interpretability resea…
ReFT: Representation Finetuning for Language Models
Zhengxuan Wu, Aryaman Arora, Zheng Wang +4
Parameter-efficient finetuning (PEFT) methods seek to adapt large neural models via updates to a small number of weights. However, much prior interpretability work has shown that r…
Mapping the Increasing Use of LLMs in Scientific Papers
Weixin Liang, Yaohui Zhang, Zhengxuan Wu +11
Scientific publishing lays the foundation of science by disseminating research findings, fostering collaboration, encouraging reproducibility, and ensuring that scientific knowledg…
pyvene: A Library for Understanding and Improving PyTorch Models via Interventions
Zhengxuan Wu, Atticus Geiger, Aryaman Arora +5
Interventions on model-internal states are fundamental operations in many areas of AI, including model editing, steering, robustness, and interpretability. To facilitate such resea…
RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representations
Jing Huang, Zhengxuan Wu, Christopher Potts +2
Individual neurons participate in the representation of multiple high-level concepts. To what extent can different interpretability methods successfully disentangle these roles? To…
A Reply to Makelov et al. (2023)'s "Interpretability Illusion" Arguments
Zhengxuan Wu, Atticus Geiger, Jing Huang +4
We respond to the recent paper by Makelov et al. (2023), which reviews subspace interchange intervention methods like distributed alignment search (DAS; Geiger et al. 2023) and cla…
Rigorously Assessing Natural Language Explanations of Neurons
Jing Huang, Atticus Geiger, Karel D'Oosterlinck +2
Natural language is an appealing medium for explaining how large language models process and store information, but evaluating the faithfulness of such explanations is challenging.…
MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions
Zexuan Zhong, Zhengxuan Wu, Christopher D. Manning +2
The information stored in large language models (LLMs) falls out of date quickly, and retraining from scratch is often not an option. This has recently given rise to a range of tec…
Interpretability at Scale: Identifying Causal Mechanisms in Alpaca
Zhengxuan Wu, Atticus Geiger, Thomas Icard +2
Obtaining human-interpretable explanations of large, general-purpose language models is an urgent goal for AI safety. However, it is just as important that our interpretability met…
ReCOGS: How Incidental Details of a Logical Form Overshadow an Evaluation of Semantic Interpretation
Zhengxuan Wu, Christopher D. Manning, Christopher Potts
Compositional generalization benchmarks for semantic parsing seek to assess whether models can accurately compute meanings for novel sentences, but operationalize this in terms of…
Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations
Atticus Geiger, Zhengxuan Wu, Christopher Potts +2
Causal abstraction is a promising theoretical framework for explainable artificial intelligence that defines when an interpretable high-level causal model is a faithful simplificat…
Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability
Atticus Geiger, Duligur Ibeling, Amir Zur +8
Causal abstraction provides a theoretical foundation for mechanistic interpretability, the field concerned with providing intelligible algorithms that are faithful simplifications…
Inducing Character-level Structure in Subword-based Language Models with Type-level Interchange Intervention Training
Jing Huang, Zhengxuan Wu, Kyle Mahowald +1
Language tasks involving character-level manipulations (e.g., spelling corrections, arithmetic operations, word games) are challenging for models operating on subword units. To add…
Causal Proxy Models for Concept-Based Model Explanations
Zhengxuan Wu, Karel D'Oosterlinck, Atticus Geiger +2
Explainability methods for NLP systems encounter a version of the fundamental problem of causal inference: for a given ground-truth input text, we never truly observe the counterfa…
ZeroC: A Neuro-Symbolic Model for Zero-shot Concept Recognition and Acquisition at Inference Time
Tailin Wu, Megan Tjandrasuwita, Zhengxuan Wu +4
Humans have the remarkable ability to recognize and acquire novel visual concepts in a zero-shot manner. Given a high-level, symbolic description of a novel concept in terms of pre…
CEBaB: Estimating the Causal Effects of Real-World Concepts on NLP Model Behavior
Eldar David Abraham, Karel D'Oosterlinck, Amir Feder +5
The increasing size and complexity of modern ML systems has improved their predictive capabilities but made their behavior harder to explain. Many techniques for model explanation…
Oolong: Investigating What Makes Transfer Learning Hard with Controlled Studies
Zhengxuan Wu, Alex Tamkin, Isabel Papadimitriou
When we transfer a pretrained language model to a new language, there are many axes of variation that change at once. To disentangle the impact of different factors like syntactic…
Causal Distillation for Language Models
Zhengxuan Wu, Atticus Geiger, Josh Rozner +5
Distillation efforts have led to language models that are more compact and efficient without serious drops in performance. The standard approach to distillation trains a student mo…
Inducing Causal Structure for Interpretable Neural Networks
Atticus Geiger, Zhengxuan Wu, Hanson Lu +5
In many areas, we have well-founded insights about causal structure that would be useful to bring into our trained models while still allowing them to learn in a data-driven fashio…
ReaSCAN: Compositional Reasoning in Language Grounding
Zhengxuan Wu, Elisa Kreiss, Desmond C. Ong +1
The ability to compositionally map language to referents, relations, and actions is an essential component of language understanding. The recent gSCAN dataset (Ruis et al. 2020, Ne…
Identifying the Limits of Cross-Domain Knowledge Transfer for Pretrained Models
Zhengxuan Wu, Nelson F. Liu, Christopher Potts
There is growing evidence that pretrained language models improve task-specific fine-tuning not just for the languages seen in pretraining, but also for new languages and even non-…