Verbalizing Multi-Token Concepts in LLMs

Xijie Gong, Tonghan Wang, Zimeng Huang

Lens methods inspect model computation by mapping intermediate activations to vocabulary tokens. Yet the concepts humans need to read out often span multiple tokens---entities, phr…