25 citations · 98 across the 17 of their papers we have counts for
6 papers · 1 filter
How to Fine-Tune Vision Models with SGD
Ananya Kumar, Ruoqi Shen, Sebastien Bubeck +1
SGD and AdamW are the two most used optimizers for fine-tuning large neural networks in computer vision. When the two methods perform the same, SGD is preferable because it uses le…
Condition-number-independent convergence rate of Riemannian Hamiltonian Monte Carlo with numerical integrators
Yunbum Kook, Yin Tat Lee, Ruoqi Shen +1
We study the convergence rate of discretized Riemannian Hamiltonian Monte Carlo on sampling from distributions in the form of on a convex body $\mathcal{M}\subset\mathb…
Private Convex Optimization in General Norms
Sivakanth Gopi, Yin Tat Lee, Daogao Liu +2
We propose a new framework for differentially private optimization of convex functions which are Lipschitz in an arbitrary norm . Our algorithms are based on a regulariz…
Data Augmentation as Feature Manipulation
Ruoqi Shen, Sébastien Bubeck, Suriya Gunasekar
Data augmentation is a cornerstone of the machine learning pipeline, yet its theoretical underpinnings remain unclear. Is it merely a way to artificially augment the data set size?…
On Optimal Early Stopping: Over-informative versus Under-informative Parametrization
Ruoqi Shen, Liyao Gao, Yi-An Ma
Early stopping is a simple and widely used method to prevent over-training neural networks. We develop theoretical results to reveal the relationship between the optimal early stop…
Sampling with Riemannian Hamiltonian Monte Carlo in a Constrained Space
Yunbum Kook, Yin Tat Lee, Ruoqi Shen +1
We demonstrate for the first time that ill-conditioned, non-smooth, constrained distributions in very high dimension, upwards of 100,000, can be sampled efficiently $\textit{in pra…