4 papers · 1 filter
Stochastic Gradients under Nuisances
Facheng Yu, Ronak Mehta, Alex Luedtke +1
Stochastic gradient optimization is the dominant learning paradigm for a variety of scenarios, from classical supervised learning to modern self-supervised learning. We consider st…
A Generalization Theory for Zero-Shot Prediction
Ronak Mehta, Zaid Harchaoui
A modern paradigm for generalization in machine learning and AI consists of pre-training a task-agnostic foundation model, generally obtained using self-supervised and multimodal c…
The Benefits of Balance: From Information Projections to Variance Reduction
Lang Liu, Ronak Mehta, Soumik Pal +1
Data balancing across multiple modalities and sources appears in various forms in foundation models in machine learning and AI, e.g. in CLIP and DINO. We show that data balancing a…
Drago: Primal-Dual Coupled Variance Reduction for Faster Distributionally Robust Optimization
Ronak Mehta, Jelena Diakonikolas, Zaid Harchaoui
We consider the penalized distributionally robust optimization (DRO) problem with a closed, convex uncertainty set, a setting that encompasses learning using -DRO and spectral/$…