69 citations · 155 across the 28 of their papers we have counts for
30 papers
Uniform-in-Time Wasserstein Stability Bounds for (Noisy) Stochastic Gradient Descent
Lingjiong Zhu, Mert Gurbuzbalaban, Anant Raj +1
Algorithmic stability is an important notion that has proven powerful for deriving generalization bounds for practical algorithms. The last decade has witnessed an increasing numbe…
Variational Principles for Mirror Descent and Mirror Langevin Dynamics
Belinda Tzen, Anant Raj, Maxim Raginsky +1
Mirror descent, introduced by Nemirovski and Yudin in the 1970s, is a primal-dual convex optimization method that can be tailored to the geometry of the optimization problem at han…
Efficient Sampling of Stochastic Differential Equations with Positive Semi-Definite Models
Anant Raj, Umut Şimşekli, Alessandro Rudi
This paper deals with the problem of efficient sampling from a stochastic differential equation, given the drift function and the diffusion matrix. The proposed approach leverages…
Algorithmic Stability of Heavy-Tailed SGD with General Loss Functions
Anant Raj, Lingjiong Zhu, Mert Gürbüzbalaban +1
Heavy-tail phenomena in stochastic gradient descent (SGD) have been reported in several empirical studies. Experimental evidence in previous works suggests a strong interplay betwe…
MAViC: Multimodal Active Learning for Video Captioning
Gyanendra Das, Xavier Thomas, Anant Raj +1
A large number of annotated video-caption pairs are required for training video captioning models, resulting in high annotation costs. Active learning can be instrumental in reduci…
Explicit Regularization in Overparametrized Models via Noise Injection
Antonio Orvieto, Anant Raj, Hans Kersting +1
Injecting noise within gradient descent has several desirable features, such as smoothing and regularizing properties. In this paper, we investigate the effects of injecting noise…
Utilising the CLT Structure in Stochastic Gradient based Sampling : Improved Analysis and Faster Algorithms
Aniket Das, Dheeraj Nagaraj, Anant Raj
We consider stochastic approximations of sampling algorithms, such as Stochastic Gradient Langevin Dynamics (SGLD) and the Random Batch Method (RBM) for Interacting Particle Dynamc…
Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squares
Anant Raj, Melih Barsbey, Mert Gürbüzbalaban +2
Recent studies have shown that heavy tails can emerge in stochastic optimization and that the heaviness of the tails have links to the generalization error. While these studies hav…
Convergence of Uncertainty Sampling for Active Learning
Anant Raj, Francis Bach
Uncertainty sampling in active learning is heavily used in practice to reduce the annotation cost. However, there has been no wide consensus on the function to be used for uncertai…
Non-stationary Online Regression
Anant Raj, Pierre Gaillard, Christophe Saad
Online forecasting under a changing environment has been a problem of increasing importance in many real-world applications. In this paper, we consider the meta-algorithm presented…
Model-specific Data Subsampling with Influence Functions
Anant Raj, Cameron Musco, Lester Mackey +1
Model selection requires repeatedly evaluating models on a given dataset and measuring their relative performances. In modern applications of machine learning, the models being con…
Causal Feature Selection via Orthogonal Search
Ashkan Soleymani, Anant Raj, Stefan Bauer +2
The problem of inferring the direct causal parents of a response variable among a large set of explanatory variables is of high practical importance in many disciplines. However, e…
Stochastic Stein Discrepancies
Jackson Gorham, Anant Raj, Lester Mackey
Stein discrepancies (SDs) monitor convergence and non-convergence in approximate inference when exact integration and sampling are intractable. However, the computation of a Stein…
Explicit Regularization of Stochastic Gradient Methods through Duality
Anant Raj, Francis Bach
We consider stochastic gradient methods under the interpolation regime where a perfect fit can be obtained (minimum loss at each observation). While previous work highlighted the i…
Importance Sampling via Local Sensitivity
Anant Raj, Cameron Musco, Lester Mackey
Given a loss function that can be written as the sum of losses over a large set of inputs , it is often desirable to approximate $…
Dual Instrumental Variable Regression
Krikamol Muandet, Arash Mehrjou, Si Kai Lee +1
We present a novel algorithm for non-linear instrumental variable (IV) regression, DualIV, which simplifies traditional two-stage methods via a dual formulation. Inspired by proble…
Orthogonal Structure Search for Efficient Causal Discovery from Observational Data
Anant Raj, Luigi Gresele, Michel Besserve +2
The problem of inferring the direct causal parents of a response variable among a large set of explanatory variables is of high practical importance in many disciplines. Recent wor…
A Differentially Private Kernel Two-Sample Test
Anant Raj, Ho Chung Leon Law, Dino Sejdinovic +1
Kernel two-sample testing is a useful statistical tool in determining whether data samples arise from different distributions without imposing any parametric assumptions on those d…
Sobolev Descent
Youssef Mroueh, Tom Sercu, Anant Raj
We study a simplification of GAN training: the problem of transporting particles from a source to a target distribution. Starting from the Sobolev GAN critic, part of the gradient…
k-SVRG: Variance Reduction for Large Scale Optimization
Anant Raj, Sebastian U. Stich
Variance reduced stochastic gradient (SGD) methods converge significantly faster than the vanilla SGD counterpart. However, these methods are not very practical on large scale prob…
On Matching Pursuit and Coordinate Descent
Francesco Locatello, Anant Raj, Sai Praneeth Karimireddy +4
Two popular examples of first-order optimization methods over linear spaces are coordinate descent and matching pursuit algorithms, with their randomized variants. While the former…
Sobolev GAN
Youssef Mroueh, Chun-Liang Li, Tom Sercu +2
We propose a new Integral Probability Metric (IPM) between distributions: the Sobolev IPM. The Sobolev IPM compares the mean discrepancy of two distributions for functions (critic)…
Safe Adaptive Importance Sampling
Sebastian U. Stich, Anant Raj, Martin Jaggi
Importance sampling has become an indispensable strategy to speed up optimization algorithms for large-scale applications. Improved adaptive variants - using importance values defi…
Approximate Steepest Coordinate Descent
Sebastian U. Stich, Anant Raj, Martin Jaggi
We propose a new selection rule for the coordinate selection in coordinate descent methods for huge-scale optimization. The efficiency of this novel scheme is provably better than…