4 papers
LLM Layers Immediately Correct Each Other
Arjun Patrawala, Jiahai Feng, Erik Jones +1
Recent methods in language model interpretability employ techniques such as sparse autoencoders to decompose residual stream contributions into linear, semantically meaningful feat…
Learning a Generative Meta-Model of LLM Activations
Grace Luo, Jiahai Feng, Trevor Darrell +2
Existing approaches for analyzing neural network activations, such as PCA and sparse autoencoders, rely on strong structural assumptions. Generative models offer an alternative: th…
Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts
Jiahai Feng, Stuart Russell, Jacob Steinhardt
Pretrained language models (LMs) can generalize to implications of facts that they are finetuned on. For example, if finetuned on ``John Doe lives in Tokyo," LMs can correctly answ…
Monitoring Latent World States in Language Models with Propositional Probes
Jiahai Feng, Stuart Russell, Jacob Steinhardt
Language models are susceptible to bias, sycophancy, backdoors, and other tendencies that lead to unfaithful responses to the input context. Interpreting internal states of languag…