7 papers
A Generalization Theory for Zero-Shot Prediction
Ronak Mehta, Zaid Harchaoui
A modern paradigm for generalization in machine learning and AI consists of pre-training a task-agnostic foundation model, generally obtained using self-supervised and multimodal c…
Supervised Stochastic Gradient Algorithms for Multi-Trial Source Separation
Ronak Mehta, Mateus Piovezan Otto, Noah Stanis +2
We develop a stochastic algorithm for independent component analysis that incorporates multi-trial supervision, which is available in many scientific contexts. The method blends a…
Stochastic Gradients under Nuisances
Facheng Yu, Ronak Mehta, Alex Luedtke +1
Stochastic gradient optimization is the dominant learning paradigm for a variety of scenarios, from classical supervised learning to modern self-supervised learning. We consider st…
Efficient Methods for Min-Max Optimization with Dual-Linear Coupling
Ronak Mehta, Jelena Diakonikolas, Zaid Harchaoui
We study a class of convex-concave min-max problems in which the coupled component of the objective is linear in at least one of the two decision vectors. We identify such problem…
Langevin Diffusion Approximation to Same Marginal Schrödinger Bridge
Medha Agarwal, Zaid Harchaoui, Garrett Mulcahy +1
We introduce a novel approximation to the same marginal Schrödinger bridge using the Langevin diffusion. As , it is known that the barycentric projection…
The Benefits of Balance: From Information Projections to Variance Reduction
Lang Liu, Ronak Mehta, Soumik Pal +1
Data balancing across multiple modalities and sources appears in various forms in foundation models in machine learning and AI, e.g. in CLIP and DINO. We show that data balancing a…
Stochastic optimization on matrices and a graphon McKean-Vlasov limit
Zaid Harchaoui, Sewoong Oh, Soumik Pal +2
We consider stochastic gradient descents on the space of large symmetric matrices of suitable functions that are invariant under permuting the rows and columns using the same permu…