5 papers
A Generalization Theory for Zero-Shot Prediction
Ronak Mehta, Zaid Harchaoui
A modern paradigm for generalization in machine learning and AI consists of pre-training a task-agnostic foundation model, generally obtained using self-supervised and multimodal c…
Supervised Stochastic Gradient Algorithms for Multi-Trial Source Separation
Ronak Mehta, Mateus Piovezan Otto, Noah Stanis +2
We develop a stochastic algorithm for independent component analysis that incorporates multi-trial supervision, which is available in many scientific contexts. The method blends a…
Stochastic Gradients under Nuisances
Facheng Yu, Ronak Mehta, Alex Luedtke +1
Stochastic gradient optimization is the dominant learning paradigm for a variety of scenarios, from classical supervised learning to modern self-supervised learning. We consider st…
Efficient Methods for Min-Max Optimization with Dual-Linear Coupling
Ronak Mehta, Jelena Diakonikolas, Zaid Harchaoui
We study a class of convex-concave min-max problems in which the coupled component of the objective is linear in at least one of the two decision vectors. We identify such problem…
The Benefits of Balance: From Information Projections to Variance Reduction
Lang Liu, Ronak Mehta, Soumik Pal +1
Data balancing across multiple modalities and sources appears in various forms in foundation models in machine learning and AI, e.g. in CLIP and DINO. We show that data balancing a…