54 citations · 200 across the 55 of their papers we have counts for
14 papers · 1 filter
From Stability to Chaos: Analyzing Gradient Descent Dynamics in Quadratic Regression
Xuxing Chen, Krishnakumar Balasubramanian, Promit Ghosal +1
We conduct a comprehensive investigation into the dynamics of gradient descent using large-order constant step-sizes in the context of quadratic regression models. Within this fram…
Zeroth-order Riemannian Averaging Stochastic Approximation Algorithms
Jiaxiang Li, Krishnakumar Balasubramanian, Shiqian Ma
We present Zeroth-order Riemannian Averaging Stochastic Approximation (\texttt{Zo-RASA}) algorithms for stochastic optimization on Riemannian manifolds. We show that \texttt{Zo-RAS…
Online covariance estimation for stochastic gradient descent under Markovian sampling
Abhishek Roy, Krishnakumar Balasubramanian
We investigate the online overlapping batch-means covariance estimator for Stochastic Gradient Descent (SGD) under Markovian sampling. Convergence rates of order $O\big(\sqrt{d}\,n…
Stochastic Nested Compositional Bi-level Optimization for Robust Feature Learning
Xuxing Chen, Krishnakumar Balasubramanian, Saeed Ghadimi
We develop and analyze stochastic approximation algorithms for solving nested compositional bi-level optimization problems. These problems involve a nested composition of poten…
Optimal Algorithms for Stochastic Bilevel Optimization under Relaxed Smoothness Conditions
Xuxing Chen, Tesi Xiao, Krishnakumar Balasubramanian
Stochastic Bilevel optimization usually involves minimizing an upper-level (UL) function that is dependent on the arg-min of a strongly-convex lower-level (LL) function. Several al…
Gaussian random field approximation via Stein's method with applications to wide random neural networks
Krishnakumar Balasubramanian, Larry Goldstein, Nathan Ross +1
We derive upper bounds on the Wasserstein distance (), with respect to -norm, between any continuous valued random field indexed by the -sphere and the…
Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descent
Tianle Liu, Promit Ghosal, Krishnakumar Balasubramanian +1
Stein Variational Gradient Descent (SVGD) is a nonparametric particle-based deterministic sampling algorithm. Despite its wide usage, understanding the theoretical properties of SV…
Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Space
Michael Diao, Krishnakumar Balasubramanian, Sinho Chewi +1
Variational inference (VI) seeks to approximate a target distribution by an element of a tractable family of distributions. Of key interest in statistics and machine learning i…
High-dimensional scaling limits and fluctuations of online least-squares SGD with smooth covariance
Krishnakumar Balasubramanian, Promit Ghosal, Ye He
We derive high-dimensional scaling limits and fluctuations for the online least-squares Stochastic Gradient Descent (SGD) algorithm by taking the properties of the data generating…
Towards a Complete Analysis of Langevin Monte Carlo: Beyond Poincaré Inequality
Alireza Mousavi-Hosseini, Tyler Farghly, Ye He +2
Langevin diffusions are rapidly convergent under appropriate functional inequality assumptions. Hence, it is natural to expect that with additional smoothness conditions to handle…
Mean-Square Analysis of Discretized Itô Diffusions for Heavy-tailed Sampling
Ye He, Tyler Farghly, Krishnakumar Balasubramanian +1
We analyze the complexity of sampling from a class of heavy-tailed distributions by discretizing a natural class of Itô diffusions associated with weighted Poincaré inequalities. B…
Improved Discretization Analysis for Underdamped Langevin Monte Carlo
Matthew Zhang, Sinho Chewi, Mufan Bill Li +2
Underdamped Langevin Monte Carlo (ULMC) is an algorithm used to sample from unnormalized densities by leveraging the momentum of a particle moving in a potential well. We provide a…
A One-Sample Decentralized Proximal Algorithm for Non-Convex Stochastic Composite Optimization
Tesi Xiao, Xuxing Chen, Krishnakumar Balasubramanian +1
We focus on decentralized stochastic non-convex optimization, where agents work together to optimize a composite objective function which is a sum of a smooth term and a non-sm…
Statistical Inference for Linear Functionals of Online SGD in High-dimensional Linear Regression
Bhavya Agrawalla, Krishnakumar Balasubramanian, Promit Ghosal
Stochastic gradient descent (SGD) has emerged as the quintessential method in a data scientist's toolbox. Using SGD for high-stakes applications requires, however, careful quantifi…