Spectral Lens: Activation and Gradient Spectra as Diagnostics of LLM Optimization

Andy Zeyi Liu, Elliot Paquette, John Sous

Training loss and throughput can hide distinct internal representation in language-model training. To examine these hidden mechanics, we use spectral measurements as practical and…