430 citations · 985 across the 29 of their papers we have counts for
31 papers
Learning Where to Steer: Noise-Space Geometry for Efficient Offline Multi-Objective Optimization with Generative Models
Yuan Lu, Esha Singh, Yi-An Ma +1
Offline multi-objective optimization (MOO) seeks solutions with better objective trade-offs using only a fixed dataset, without querying the objectives. Diffusion models trained on…
Reverse Diffusion Monte Carlo
Xunpeng Huang, Hanze Dong, Yifan Hao +2
We propose a Monte Carlo sampler from the reverse diffusion process. Unlike the practice of diffusion models, where the intermediary updates -- the score functions -- are learned w…
Langevin Thompson Sampling with Logarithmic Communication: Bandits and Reinforcement Learning
Amin Karbasi, Nikki Lijing Kuang, Yi-An Ma +1
Thompson sampling (TS) is widely used in sequential decision making due to its ease of use and appealing empirical performance. However, many existing analytical and empirical resu…
Aiming towards the minimizers: fast convergence of SGD for overparametrized problems
Chaoyue Liu, Dmitriy Drusvyatskiy, Mikhail Belkin +2
Modern machine learning paradigms, such as deep learning, occur in or close to the interpolation regime, wherein the number of model parameters is much larger than the number of da…
Disentangled Multi-Fidelity Deep Bayesian Active Learning
Dongxia Wu, Ruijia Niu, Matteo Chinazzi +2
To balance quality and cost, various domain areas of science and engineering run simulations at multiple levels of sophistication. Multi-fidelity active learning aims to learn a di…
On the Convergence of Black-Box Variational Inference
Kyurae Kim, Jisu Oh, Kaiwen Wu +2
We provide the first convergence guarantee for full black-box variational inference (BBVI), also known as Monte Carlo variational inference. While preliminary investigations worked…
Statistical and Computational Trade-offs in Variational Inference: A Case Study in Inferential Model Selection
Kush Bhatia, Nikki Lijing Kuang, Yi-An Ma +1
Variational inference has recently emerged as a popular alternative to the classical Markov chain Monte Carlo (MCMC) in large-scale Bayesian inference. The core idea is to trade st…
Multi-fidelity Hierarchical Neural Processes
Dongxia Wu, Matteo Chinazzi, Alessandro Vespignani +2
Science and engineering fields use computer simulation extensively. These simulations are often run at multiple levels of sophistication to balance accuracy and efficiency. Multi-f…
Robust Calibration with Multi-domain Temperature Scaling
Yaodong Yu, Stephen Bates, Yi Ma +1
Uncertainty quantification is essential for the reliable deployment of machine learning models to high-stakes application domains. Uncertainty quantification is all the more challe…
Dimension Independent Generalization of DP-SGD for Overparameterized Smooth Convex Optimization
Yi-An Ma, Teodor Vanislavov Marinov, Tong Zhang
This paper considers the generalization performance of differentially private convex learning. We demonstrate that the convergence analysis of Langevin algorithms can be used to ob…
On Optimal Early Stopping: Over-informative versus Under-informative Parametrization
Ruoqi Shen, Liyao Gao, Yi-An Ma
Early stopping is a simple and widely used method to prevent over-training neural networks. We develop theoretical results to reveal the relationship between the optimal early stop…
On Convergence of Federated Averaging Langevin Dynamics
Wei Deng, Qian Zhang, Yi-An Ma +2
We propose a federated averaging Langevin algorithm (FA-LD) for uncertainty quantification and mean predictions with distributed clients. In particular, we generalize beyond normal…
When is the Convergence Time of Langevin Algorithms Dimension Independent? A Composite Optimization Viewpoint
Yoav Freund, Yi-An Ma, Tong Zhang
There has been a surge of works bridging MCMC sampling and optimization, with a specific focus on translating non-asymptotic convergence guarantees for optimization problems into t…
Variational Refinement for Importance Sampling Using the Forward Kullback-Leibler Divergence
Ghassen Jerfel, Serena Wang, Clara Fannjiang +3
Variational Inference (VI) is a popular alternative to asymptotically exact sampling in Bayesian inference. Its main workhorse is optimization over a reverse Kullback-Leibler diver…
Quantifying Uncertainty in Deep Spatiotemporal Forecasting
Dongxia Wu, Liyao Gao, Xinyue Xiong +4
Deep learning is gaining increasing popularity for spatiotemporal forecasting. However, prior works have mostly focused on point estimates without quantifying the uncertainty of th…
Deep Bayesian Active Learning for Accelerating Stochastic Simulation
Dongxia Wu, Ruijia Niu, Matteo Chinazzi +3
Stochastic simulations such as large-scale, spatiotemporal, age-structured epidemic models are computationally expensive at fine-grained resolution. While deep surrogate models can…
DeepGLEAM: A hybrid mechanistic and deep learning model for COVID-19 forecasting
Dongxia Wu, Liyao Gao, Xinyue Xiong +4
We introduce DeepGLEAM, a hybrid model for COVID-19 forecasting. DeepGLEAM combines a mechanistic stochastic simulation model GLEAM with deep learning. It uses deep learning to lea…
Underspecification Presents Challenges for Credibility in Modern Machine Learning
Alexander D'Amour, Katherine Heller, Dan Moldovan +37
ML models often exhibit unexpectedly poor behavior when they are deployed in real-world domains. We identify underspecification as a key reason for these failures. An ML pipeline i…
Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors
Michael W. Dusenberry, Ghassen Jerfel, Yeming Wen +5
Bayesian neural networks (BNNs) demonstrate promising success in improving the robustness and uncertainty quantification of modern deep learning. However, they generally struggle w…
On Thompson Sampling with Langevin Algorithms
Eric Mazumdar, Aldo Pacchiano, Yi-an Ma +2
Thompson sampling for multi-armed bandit problems is known to enjoy favorable performance in both theory and practice. However, it suffers from a significant limitation computation…
High-Order Langevin Diffusion Yields an Accelerated MCMC Algorithm
Wenlong Mou, Yi-An Ma, Martin J. Wainwright +2
We propose a Markov chain Monte Carlo (MCMC) algorithm based on third-order Langevin dynamics for sampling from distributions with log-concave and smooth densities. The higher-orde…
Bayesian Robustness: A Nonasymptotic Viewpoint
Kush Bhatia, Yi-An Ma, Anca D. Dragan +2
We study the problem of robustly estimating the posterior distribution for the setting where observed data can be contaminated with potentially adversarial outliers. We propose Rob…
Is There an Analog of Nesterov Acceleration for MCMC?
Yi-An Ma, Niladri Chatterji, Xiang Cheng +3
We formulate gradient-based Markov chain Monte Carlo (MCMC) sampling as optimization on the space of probability measures, with Kullback-Leibler (KL) divergence as the objective fu…
Sampling Can Be Faster Than Optimization
Yi-An Ma, Yuansi Chen, Chi Jin +2
Optimization algorithms and Monte Carlo sampling algorithms have provided the computational foundations for the rapid growth in applications of statistical machine learning in rece…