8 citations · 18 across the 7 of their papers we have counts for
Showing 2026Show all
2 papers · 1 filter
cs.LG2026
(How) Learning Rates Regulate Catastrophic Overtraining
Mark Rofin, Aditya Varre, Nicolas Flammarion
Supervised fine-tuning (SFT) is a common first stage of LLM post-training, teaching the model to follow instructions and shaping its behavior as a helpful assistant. At the same ti…
cs.LG2026
Gradient Flow Polarizes Softmax Outputs towards Low-Entropy Solutions
Aditya Varre, Mark Rofin, Nicolas Flammarion
Understanding the intricate non-convex training dynamics of softmax-based models is crucial for explaining the empirical success of transformers. In this article, we analyze the gr…