105 citations · 271 across the 24 of their papers we have counts for
26 papers
High-accuracy and dimension-free sampling with diffusions
Khashayar Gatmiry, Sitan Chen, Adil Salim
Diffusion models have shown remarkable empirical success in sampling from rich multi-modal distributions. Their inference relies on numerically solving a certain differential equat…
Long-time asymptotics of noisy SVGD outside the population limit
Victor Priser, Pascal Bianchi, Adil Salim
Stein Variational Gradient Descent (SVGD) is a widely used sampling algorithm that has been successfully applied in several areas of Machine Learning. SVGD operates by iteratively…
Gaussian random field approximation via Stein's method with applications to wide random neural networks
Krishnakumar Balasubramanian, Larry Goldstein, Nathan Ross +1
We derive upper bounds on the Wasserstein distance (), with respect to -norm, between any continuous valued random field indexed by the -sphere and the…
Textbooks Are All You Need
Suriya Gunasekar, Yi Zhang, Jyoti Aneja +16
We introduce phi-1, a new large language model for code, with significantly smaller size than competing models: phi-1 is a Transformer-based model with 1.3B parameters, trained for…
The probability flow ODE is provably fast
Sitan Chen, Sinho Chewi, Holden Lee +3
We provide the first polynomial-time convergence guarantees for the probability flow ODE implementation (together with a corrector step) of score-based generative modeling. Our ana…
Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Space
Michael Diao, Krishnakumar Balasubramanian, Sinho Chewi +1
Variational inference (VI) seeks to approximate a target distribution by an element of a tractable family of distributions. Of key interest in statistics and machine learning i…
Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions
Sitan Chen, Sinho Chewi, Jerry Li +3
We provide theoretical convergence guarantees for score-based generative models (SGMs) such as denoising diffusion probabilistic models (DDPMs), which constitute the backbone of la…
On the complexity of finding stationary points of smooth functions in one dimension
Sinho Chewi, Sébastien Bubeck, Adil Salim
We characterize the query complexity of finding stationary points of one-dimensional non-convex but smooth functions. We consider four settings, based on whether the algorithms und…
Federated Learning with a Sampling Algorithm under Isoperimetry
Lukang Sun, Adil Salim, Peter Richtárik
Federated learning uses a set of techniques to efficiently distribute the training of a machine learning algorithm across several devices, who own the training data. These techniqu…
Improved analysis for a proximal algorithm for sampling
Yongxin Chen, Sinho Chewi, Adil Salim +1
We study the proximal sampler of Lee, Shen, and Tian (2021) and obtain new convergence guarantees under weaker assumptions than strong log-concavity: namely, our results hold for (…
Towards a Theory of Non-Log-Concave Sampling: First-Order Stationarity Guarantees for Langevin Monte Carlo
Krishnakumar Balasubramanian, Sinho Chewi, Murat A. Erdogdu +2
For the task of sampling from a density on , where is possibly non-convex but -gradient Lipschitz, we prove that averaged Langevin Monte Ca…
A Convergence Theory for SVGD in the Population Limit under Talagrand's Inequality T1
Adil Salim, Lukang Sun, Peter Richtárik
Stein Variational Gradient Descent (SVGD) is an algorithm for sampling from a target density which is known up to a multiplicative constant. Although SVGD is a popular algorithm in…
An Optimal Algorithm for Strongly Convex Minimization under Affine Constraints
Adil Salim, Laurent Condat, Dmitry Kovalev +1
Optimization problems under affine constraints appear in various areas of machine learning. We consider the task of minimizing a smooth strongly convex function F(x) under the affi…
Optimal and Practical Algorithms for Smooth and Strongly Convex Decentralized Optimization
Dmitry Kovalev, Adil Salim, Peter Richtárik
We consider the task of decentralized minimization of the sum of smooth strongly convex functions stored across the nodes of a network. For this problem, lower bounds on the number…
A Non-Asymptotic Analysis for Stein Variational Gradient Descent
Anna Korba, Adil Salim, Michael Arbel +2
We study the Stein Variational Gradient Descent (SVGD) algorithm, which optimises a set of particles to approximate a target probability distribution on $\mathbb{…
Primal Dual Interpretation of the Proximal Stochastic Gradient Langevin Algorithm
Adil Salim, Peter Richtárik
We consider the task of sampling with respect to a log concave probability distribution. The potential of the target distribution is assumed to be composite, \textit{i.e.}, written…
Dualize, Split, Randomize: Toward Fast Nonsmooth Optimization Algorithms
Adil Salim, Laurent Condat, Konstantin Mishchenko +1
We consider minimizing the sum of three convex functions, where the first one F is smooth, the second one is nonsmooth and proximable and the third one is the composition of a nons…
Distributed Fixed Point Methods with Compressed Iterates
Sélim Chraibi, Ahmed Khaled, Dmitry Kovalev +3
We propose basic and natural assumptions under which iterative optimization methods with compressed iterates can be analyzed. This problem is motivated by the practice of federated…
A Strong Law of Large Numbers for Random Monotone Operators
Adil Salim
Random monotone operators are stochastic versions of maximal monotone operators which play an important role in stochastic nonsmooth optimization. Several stochastic nonsmooth opti…
Maximum Mean Discrepancy Gradient Flow
Michael Arbel, Anna Korba, Adil Salim +1
We construct a Wasserstein gradient flow of the maximum mean discrepancy (MMD) and study its convergence properties. The MMD is an integral probability metric defined for a reprodu…
Stochastic Proximal Langevin Algorithm: Potential Splitting and Nonasymptotic Rates
Adil Salim, Dmitry Kovalev, Peter Richtárik
We propose a new algorithm---Stochastic Proximal Langevin Algorithm (SPLA)---for sampling from a log concave distribution. Our method is a generalization of the Langevin algorithm…
A Fully Stochastic Primal-Dual Algorithm
Pascal Bianchi, Walid Hachem, Adil Salim
A new stochastic primal--dual algorithm for solving a composite optimization problem is proposed. It is assumed that all the functions/operators that enter the optimization problem…
A Constant Step Stochastic Douglas-Rachford Algorithm with Application to Non Separable Regularizations
Adil Salim, Pascal Bianchi, Walid Hachem
The Douglas Rachford algorithm is an algorithm that converges to a minimizer of a sum of two convex functions. The algorithm consists in fixed point iterations involving computatio…
Snake: a Stochastic Proximal Gradient Algorithm for Regularized Problems over Large Graphs
Adil Salim, Pascal Bianchi, Walid Hachem
A regularized optimization problem over a large unstructured graph is studied, where the regularization term is tied to the graph geometry. Typical regularization examples include…