Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Learning a Generative Meta-Model of LLM Activations
Grace Luo, Jiahai Feng, Trevor Darrell +2
Existing approaches for analyzing neural network activations, such as PCA and sparse autoencoders, rely on strong structural assumptions. Generative models offer an alternative: th…
cs.LG2024
Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts
Jiahai Feng, Stuart Russell, Jacob Steinhardt
Pretrained language models (LMs) can generalize to implications of facts that they are finetuned on. For example, if finetuned on ``John Doe lives in Tokyo," LMs can correctly answ…