54 citations · 200 across the 55 of their papers we have counts for
56 papers
Mirror Flow Matching with Heavy-Tailed Priors for Generative Modeling on Convex Domains
Yunrui Guan, Krishnakumar Balasubramanian, Shiqian Ma
We study generative modeling on convex domains using flow matching and mirror maps, and identify two fundamental challenges. First, standard log-barrier mirror maps induce heavy-ta…
Riemannian Proximal Sampler for High-accuracy Sampling on Manifolds
Yunrui Guan, Krishnakumar Balasubramanian, Shiqian Ma
We introduce the Riemannian Proximal Sampler, a method for sampling from densities defined on Riemannian manifolds. The performance of this sampler critically depends on two key or…
Online Covariance Estimation in Nonsmooth Stochastic Approximation
Liwei Jiang, Abhishek Roy, Krishna Balasubramanian +3
We consider applying stochastic approximation (SA) methods to solve nonsmooth variational inclusion problems. Existing studies have shown that the averaged iterates of SA methods e…
Stochastic Optimization Algorithms for Instrumental Variable Regression with Streaming Data
Xuxing Chen, Abhishek Roy, Yifan Hu +1
We develop and analyze algorithms for instrumental variable regression by viewing the problem as a conditional stochastic optimization problem. In the context of least-squares inst…
A Separation in Heavy-Tailed Sampling: Gaussian vs. Stable Oracles for Proximal Samplers
Ye He, Alireza Mousavi-Hosseini, Krishnakumar Balasubramanian +1
We study the complexity of heavy-tailed sampling and present a separation result in terms of obtaining high-accuracy versus low-accuracy guarantees i.e., samplers that require only…
From Stability to Chaos: Analyzing Gradient Descent Dynamics in Quadratic Regression
Xuxing Chen, Krishnakumar Balasubramanian, Promit Ghosal +1
We conduct a comprehensive investigation into the dynamics of gradient descent using large-order constant step-sizes in the context of quadratic regression models. Within this fram…
Zeroth-order Riemannian Averaging Stochastic Approximation Algorithms
Jiaxiang Li, Krishnakumar Balasubramanian, Shiqian Ma
We present Zeroth-order Riemannian Averaging Stochastic Approximation (\texttt{Zo-RASA}) algorithms for stochastic optimization on Riemannian manifolds. We show that \texttt{Zo-RAS…
Online covariance estimation for stochastic gradient descent under Markovian sampling
Abhishek Roy, Krishnakumar Balasubramanian
We investigate the online overlapping batch-means covariance estimator for Stochastic Gradient Descent (SGD) under Markovian sampling. Convergence rates of order $O\big(\sqrt{d}\,n…
Stochastic Nested Compositional Bi-level Optimization for Robust Feature Learning
Xuxing Chen, Krishnakumar Balasubramanian, Saeed Ghadimi
We develop and analyze stochastic approximation algorithms for solving nested compositional bi-level optimization problems. These problems involve a nested composition of poten…
Optimal Algorithms for Stochastic Bilevel Optimization under Relaxed Smoothness Conditions
Xuxing Chen, Tesi Xiao, Krishnakumar Balasubramanian
Stochastic Bilevel optimization usually involves minimizing an upper-level (UL) function that is dependent on the arg-min of a strongly-convex lower-level (LL) function. Several al…
Gaussian random field approximation via Stein's method with applications to wide random neural networks
Krishnakumar Balasubramanian, Larry Goldstein, Nathan Ross +1
We derive upper bounds on the Wasserstein distance (), with respect to -norm, between any continuous valued random field indexed by the -sphere and the…
Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descent
Tianle Liu, Promit Ghosal, Krishnakumar Balasubramanian +1
Stein Variational Gradient Descent (SVGD) is a nonparametric particle-based deterministic sampling algorithm. Despite its wide usage, understanding the theoretical properties of SV…
Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Space
Michael Diao, Krishnakumar Balasubramanian, Sinho Chewi +1
Variational inference (VI) seeks to approximate a target distribution by an element of a tractable family of distributions. Of key interest in statistics and machine learning i…
High-dimensional scaling limits and fluctuations of online least-squares SGD with smooth covariance
Krishnakumar Balasubramanian, Promit Ghosal, Ye He
We derive high-dimensional scaling limits and fluctuations for the online least-squares Stochastic Gradient Descent (SGD) algorithm by taking the properties of the data generating…
Towards a Complete Analysis of Langevin Monte Carlo: Beyond Poincaré Inequality
Alireza Mousavi-Hosseini, Tyler Farghly, Ye He +2
Langevin diffusions are rapidly convergent under appropriate functional inequality assumptions. Hence, it is natural to expect that with additional smoothness conditions to handle…
Mean-Square Analysis of Discretized Itô Diffusions for Heavy-tailed Sampling
Ye He, Tyler Farghly, Krishnakumar Balasubramanian +1
We analyze the complexity of sampling from a class of heavy-tailed distributions by discretizing a natural class of Itô diffusions associated with weighted Poincaré inequalities. B…
Improved Discretization Analysis for Underdamped Langevin Monte Carlo
Matthew Zhang, Sinho Chewi, Mufan Bill Li +2
Underdamped Langevin Monte Carlo (ULMC) is an algorithm used to sample from unnormalized densities by leveraging the momentum of a particle moving in a potential well. We provide a…
A One-Sample Decentralized Proximal Algorithm for Non-Convex Stochastic Composite Optimization
Tesi Xiao, Xuxing Chen, Krishnakumar Balasubramanian +1
We focus on decentralized stochastic non-convex optimization, where agents work together to optimize a composite objective function which is a sum of a smooth term and a non-sm…
Statistical Inference for Linear Functionals of Online SGD in High-dimensional Linear Regression
Bhavya Agrawalla, Krishnakumar Balasubramanian, Promit Ghosal
Stochastic gradient descent (SGD) has emerged as the quintessential method in a data scientist's toolbox. Using SGD for high-stakes applications requires, however, careful quantifi…
Regularized Stein Variational Gradient Flow
Ye He, Krishnakumar Balasubramanian, Bharath K. Sriperumbudur +1
The Stein Variational Gradient Descent (SVGD) algorithm is a deterministic particle method for sampling. However, a mean-field analysis reveals that the gradient flow corresponding…
Stochastic Zeroth-order Functional Constrained Optimization: Oracle Complexity and Applications
Anthony Nguyen, Krishnakumar Balasubramanian
Functionally constrained stochastic optimization problems, where neither the objective function nor the constraint functions are analytically available, arise frequently in machine…
Decentralized Stochastic Bilevel Optimization with Improved per-Iteration Complexity
Xuxing Chen, Minhui Huang, Shiqian Ma +1
Bilevel optimization recently has received tremendous attention due to its great success in solving important machine learning problems like meta learning, reinforcement learning,…
Constrained Stochastic Nonconvex Optimization with State-dependent Markov Data
Abhishek Roy, Krishnakumar Balasubramanian, Saeed Ghadimi
We study stochastic optimization algorithms for constrained nonconvex stochastic optimization problems with Markovian data. In particular, we focus on the case when the transition…
Functional linear and single-index models: A unified approach via Gaussian Stein identity
Krishnakumar Balasubramanian, Hans-Georg Müller, Bharath K. Sriperumbudur
Functional linear and single-index models are core regression methods in functional data analysis and are widely used for performing regression in a wide range of applications when…