37 citations · 63 across the 28 of their papers we have counts for
4 papers · 1 filter
PreFT: Prefill-only finetuning for efficient inference
Andrew Lanpouthakoun, Aryaman Arora, Zhengxuan Wu +4
Large language models can now be personalised efficiently at scale using parameter efficient finetuning methods (PEFTs), but serving user-specific PEFTs harms throughput, even with…
pyvene: A Library for Understanding and Improving PyTorch Models via Interventions
Zhengxuan Wu, Atticus Geiger, Aryaman Arora +5
Interventions on model-internal states are fundamental operations in many areas of AI, including model editing, steering, robustness, and interpretability. To facilitate such resea…
A Reply to Makelov et al. (2023)'s "Interpretability Illusion" Arguments
Zhengxuan Wu, Atticus Geiger, Jing Huang +4
We respond to the recent paper by Makelov et al. (2023), which reviews subspace interchange intervention methods like distributed alignment search (DAS; Geiger et al. 2023) and cla…
Localizing Model Behavior with Path Patching
Nicholas Goldowsky-Dill, Chris MacLeod, Lucas Sato +1
Localizing behaviors of neural networks to a subset of the network's components or a subset of interactions between components is a natural first step towards analyzing network mec…