2 papers
cs.LG2023
SAM operates far from home: eigenvalue regularization as a dynamical phenomenon
Atish Agarwala, Yann N. Dauphin
The Sharpness Aware Minimization (SAM) optimization algorithm has been shown to control large eigenvalues of the loss Hessian and provide generalization benefits in a variety of se…
cs.LG2022
Deep equilibrium networks are sensitive to initialization statistics
Atish Agarwala, Samuel S. Schoenholz
Deep equilibrium networks (DEQs) are a promising way to construct models which trade off memory for compute. However, theoretical understanding of these models is still lacking com…