42 citations · 156 across the 35 of their papers we have counts for
22 papers · 1 filter
Efficiently learning and sampling multimodal distributions with data-based initialization
Frederic Koehler, Holden Lee, Thuy-Duong Vuong
We consider the problem of sampling a multimodal distribution with a Markov chain given a small number of samples from the stationary measure. Although mixing can be arbitrarily sl…
What does guidance do? A fine-grained analysis in a simple setting
Muthu Chidambaram, Khashayar Gatmiry, Sitan Chen +2
The use of guidance in diffusion models was originally motivated by the premise that the guidance-modified score is that of the data distribution tilted by a conditional likelihood…
Learning Mixtures of Gaussians Using Diffusion Models
Khashayar Gatmiry, Jonathan Kelner, Holden Lee
We give a new algorithm for learning mixtures of Gaussians (with identity covariance in ) to TV error , with quasi-polynomial ($O(n^{\text{poly\,log}…
How Flawed Is ECE? An Analysis via Logit Smoothing
Muthu Chidambaram, Holden Lee, Colin McSwiggen +1
Informally, a model is calibrated if its predictions are correct with a probability that matches the confidence of the prediction. By far the most common method in the literature f…
Provable benefits of score matching
Chirag Pabbaraju, Dhruv Rohatgi, Anish Sevekari +3
Score matching is an alternative to maximum likelihood (ML) for estimating a probability distribution parametrized up to a constant of proportionality. By fitting the ''score'' of…
The probability flow ODE is provably fast
Sitan Chen, Sinho Chewi, Holden Lee +3
We provide the first polynomial-time convergence guarantees for the probability flow ODE implementation (together with a corrector step) of score-based generative modeling. Our ana…
Improved Bound for Mixing Time of Parallel Tempering
Holden Lee, Zeyu Shen
In the field of sampling algorithms, MCMC (Markov Chain Monte Carlo) methods are widely used when direct sampling is not possible. However, multimodality of target distributions of…
Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions
Hongrui Chen, Holden Lee, Jianfeng Lu
We give an improved theoretical analysis of score-based generative modeling. Under a score estimate with small error (averaged across timesteps), we provide efficient converg…
Convergence of score-based generative modeling for general data distributions
Holden Lee, Jianfeng Lu, Yixin Tan
Score-based generative modeling (SGM) has grown to be a hugely successful method for learning to generate samples from complex data distributions such as that of images and audio.…
Pitfalls of Gaussians as a noise distribution in NCE
Holden Lee, Chirag Pabbaraju, Anish Sevekari +1
Noise Contrastive Estimation (NCE) is a popular approach for learning probability density functions parameterized up to a constant of proportionality. The main idea is to design a…
Convergence for score-based generative modeling with polynomial complexity
Holden Lee, Jianfeng Lu, Yixin Tan
Score-based generative modeling (SGM) is a highly successful approach for learning a probability distribution from data and generating further samples. We prove the first polynomia…
Universal Approximation for Log-concave Distributions using Well-conditioned Normalizing Flows
Holden Lee, Chirag Pabbaraju, Anish Sevekari +1
Normalizing flows are a widely used class of latent-variable generative models with a tractable likelihood. Affine-coupling (Dinh et al, 2014-16) models are a particularly common t…
Improved rates for prediction and identification of partially observed linear dynamical systems
Holden Lee
Identification of a linear time-invariant dynamical system from partial observations is a fundamental problem in control theory. Particularly challenging are systems exhibiting lon…
Efficient sampling from the Bingham distribution
Rong Ge, Holden Lee, Jianfeng Lu +1
We give a algorithm for exact sampling from the Bingham distribution on the sphere with expected runtime of $\operatorname{poly}(d…
No-Regret Prediction in Marginally Stable Systems
Udaya Ghai, Holden Lee, Karan Singh +2
We consider the problem of online prediction in a marginally stable linear dynamical system subject to bounded adversarial or (non-isotropic) stochastic perturbations. This poses t…
Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets
Rohith Kuditipudi, Xiang Wang, Holden Lee +5
Mode connectivity is a surprising phenomenon in the loss landscape of deep nets. Optima -- at least those discovered by gradient-based optimization -- turn out to be connected by s…
Robust guarantees for learning an autoregressive filter
Holden Lee, Cyril Zhang
The optimal predictor for a linear dynamical system (with hidden state and Gaussian noise) takes the form of an autoregressive linear filter, namely the Kalman filter. However, a f…
Online Sampling from Log-Concave Distributions
Holden Lee, Oren Mangoubi, Nisheeth K. Vishnoi
Given a sequence of convex functions , we study the problem of sampling from the Gibbs distribution for each epoch in…
Simulated Tempering Langevin Monte Carlo II: An Improved Proof using Soft Markov Chain Decomposition
Rong Ge, Holden Lee, Andrej Risteski
A key task in Bayesian machine learning is sampling from distributions that are only specified up to a partition function (i.e., constant of proportionality). One prevalent example…
Spectral Filtering for General Linear Dynamical Systems
Elad Hazan, Holden Lee, Karan Singh +2
We give a polynomial-time algorithm for learning latent-state linear dynamical systems without system identification, and without assumptions on the spectral radius of the system's…
Beyond Log-concavity: Provable Guarantees for Sampling Multi-modal Distributions using Simulated Tempering Langevin Monte Carlo
Rong Ge, Holden Lee, Andrej Risteski
A key task in Bayesian statistics is sampling from distributions that are only specified up to a partition function (i.e., constant of proportionality). However, without any assump…
On the ability of neural nets to express distributions
Holden Lee, Rong Ge, Tengyu Ma +2
Deep neural nets have caused a revolution in many classification tasks. A related ongoing revolution -- also theoretically not understood -- concerns their ability to serve as gene…