Tracking the Feature Dynamics in LLM Training: A Mechanistic Study

Yang Xu, Yi Wang, Hengguan Huang +1

Understanding training dynamics and feature evolution is crucial for the mechanistic interpretability of large language models (LLMs). Although sparse autoencoders (SAEs) have been…