CAST: Compositional Analysis via Spectral Tracking for Understanding Transformer Layer Functions

Zihao Fu, Ming Liao, Chris Russell +1

Large language models have achieved remarkable success but remain largely black boxes with poorly understood internal mechanisms. To address this limitation, many researchers have…