2 papers
cs.LG2026
q0: Primitives for Hyper-Epoch Pretraining
Bishwas Mandal, Shmuel Berman, Akshay Vegesna +1
Multi-epoch training is becoming the standard now that compute is growing faster than the supply of high-quality text. But pretraining a single model saturates within a few passes,…
cs.LG2025
Decoupling Search and Learning in Neural Net Training
Akshay Vegesna, Samip Dahal
Gradient descent typically converges to a single minimum of the training loss without mechanisms to explore alternative minima that may generalize better. Searching for diverse min…