Challenges with unsupervised LLM knowledge discovery

Sebastian Farquhar, Vikrant Varma, Zachary Kenton +3

We show that existing unsupervised methods on large language model (LLM) activations do not discover knowledge -- instead they seem to discover whatever feature of the activations…