Guiding LLM Post-training Data Engineering with Model Internals from Sparse Autoencoders

Yi Jing, Zao Dai, Jinwu Hu +4

Model internals encode rich information about how a large language model (LLM) processes its training data; however, post-training data engineering largely relies on external signa…