2 citations · 2 across the 2 of their papers we have counts for
2 papers
cs.LG2024
Towards Quantifying the Preconditioning Effect of Adam
Rudrajit Das, Naman Agarwal, Sujay Sanghavi +1
There is a notable dearth of results characterizing the preconditioning effect of Adam and showing how it may alleviate the curse of ill-conditioning -- an issue plaguing gradient…
cs.LG2023★ 2 cited
Understanding Self-Distillation in the Presence of Label Noise
Rudrajit Das, Sujay Sanghavi
Self-distillation (SD) is the process of first training a \enquote{teacher} model and then using its predictions to train a \enquote{student} model with the \textit{same} architect…