5 papers · 1 filter
A Generalization Theory for Zero-Shot Prediction
Ronak Mehta, Zaid Harchaoui
A modern paradigm for generalization in machine learning and AI consists of pre-training a task-agnostic foundation model, generally obtained using self-supervised and multimodal c…
Stochastic Gradients under Nuisances
Facheng Yu, Ronak Mehta, Alex Luedtke +1
Stochastic gradient optimization is the dominant learning paradigm for a variety of scenarios, from classical supervised learning to modern self-supervised learning. We consider st…
The Benefits of Balance: From Information Projections to Variance Reduction
Lang Liu, Ronak Mehta, Soumik Pal +1
Data balancing across multiple modalities and sources appears in various forms in foundation models in machine learning and AI, e.g. in CLIP and DINO. We show that data balancing a…
Drago: Primal-Dual Coupled Variance Reduction for Faster Distributionally Robust Optimization
Ronak Mehta, Jelena Diakonikolas, Zaid Harchaoui
We consider the penalized distributionally robust optimization (DRO) problem with a closed, convex uncertainty set, a setting that encompasses learning using -DRO and spectral/$…
Distributionally Robust Optimization with Bias and Variance Reduction
Ronak Mehta, Vincent Roulet, Krishna Pillutla +1
We consider the distributionally robust optimization (DRO) problem with spectral risk-based uncertainty set and -divergence penalty. This formulation includes common risk-sensit…