2 papers
cs.LG2026
Force without transmission: a depth-induced rank collapse that no loss on the representation reopens
Martin Hofmann, Patrick Mäder
Training can drive a transformer into a rank collapse: all token representations point in one direction, and learning stops. In a related collapse of attention, a loss term with a…
cs.LG2026
When does a network's training history predict its future learning better than its current state? Evidence from a response probe and a forecasting screen
Martin Hofmann, Patrick Mäder
Networks that behave alike now can still learn differently when training continues. Work on loss of plasticity and critical periods shows that the path to a state shapes what follo…