The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
arXiv:1111.4246
Abstract
Hamiltonian Monte Carlo (HMC) is a Markov chain Monte Carlo (MCMC) algorithm that avoids the random walk behavior and sensitivity to correlated parameters that plague many MCMC methods by taking a series of steps informed by first-order gradient information. These features allow it to converge to high-dimensional target distributions much more quickly than simpler methods such as random walk Metropolis or Gibbs sampling. However, HMC's performance is highly sensitive to two user-specified parameters: a step size ε and a desired number of steps L. In particular, if L is too small then the algorithm exhibits undesirable random walk behavior, while if L is too large the algorithm wastes computation. We introduce the No-U-Turn Sampler (NUTS), an extension to HMC that eliminates the need to set a number of steps L. NUTS uses a recursive algorithm to build a set of likely candidate points that spans a wide swath of the target distribution, stopping automatically when it starts to double back and retrace its steps. Empirically, NUTS perform at least as efficiently as and sometimes more efficiently than a well tuned standard HMC method, without requiring user intervention or costly tuning runs. We also derive a method for adapting the step size parameter ε on the fly based on primal-dual averaging. NUTS can thus be used with no hand-tuning at all. NUTS is also suitable for applications such as BUGS-style automatic inference engines that require efficient "turnkey" sampling algorithms.
30 pages, 7 figures
References in corpus (1)
Cited by in corpus (178)
- Variational Inference: A Review for Statisticians
- dynesty: A Dynamic Nested Sampling Package for Estimating Bayesian Posteriors and Evidences
- Rank-normalization, folding, and localization: An improved for assessing convergence of MCMC
- Inferring change points in the COVID-19 spreading reveals the effectiveness of interventions
- Sparsity information and regularization in the horseshoe and other shrinkage priors
- Stochastic Gradient Hamiltonian Monte Carlo
- A Review of automatic differentiation and its efficient implementation
- Obscuration-dependent evolution of Active Galactic Nuclei
- Validating Bayesian Inference Algorithms with Simulation-Based Calibration
- The pattern speed of the Milky Way bar from transverse velocities
- Isochrone ages for ~3 million stars with the second Gaia data release
- Most of the photons that reionized the Universe came from dwarf galaxies
- A radio parallax to the black hole X-ray binary MAXI J1820+070
- Detect and Correct Bias in Multi-Site Neuroimaging Datasets
- Snake in the Clouds: A new nearby dwarf galaxy in the Magellanic bridge
- Nested Sampling Methods
- On the Geometric Ergodicity of Hamiltonian Monte Carlo
- ALMA Characterises the Dust Temperature of z ~ 5.5 Star-Forming Galaxies
- Improving Simulation Efficiency of MCMC for Inverse Modeling of Hydrologic Systems with a Kalman-Inspired Proposal Distribution
- The cosmology dependence of galaxy clustering and lensing from a hybrid -body-perturbation theory model
- Randomized Hamiltonian Monte Carlo
- HypoSVI: Hypocenter inversion with Stein variational inference and Physics Informed Neural Networks
- Scaling Exact Inference for Discrete Probabilistic Programs
- A critical review of statistical calibration/prediction models handling data inconsistency and model inadequacy
- Investigating hot-Jupiter inflated radii with hierarchical Bayesian modelling
- DESI 2024 V: Full-Shape Galaxy Clustering from Galaxies and Quasars
- Mapping the Galactic Metallicity Gradient with Open Clusters: The State-of-the-Art and Future Challenges
- Coupling and Convergence for Hamiltonian Monte Carlo
- Monitoring the COVID-19 epidemic with nationwide telecommunication data
- On the kinematics of a runaway Be star population
- Transverse kinematics of the Galactic bar-bulge from VVV and Gaia
- Spatial Bayesian hierarchical modeling of precipitation extremes over a large domain
- Storywrangler: A massive exploratorium for sociolinguistic, cultural, socioeconomic, and political timelines using Twitter
- A Bayesian direct method implementation to fit emission line spectra: Application to the primordial He abundance determination
- Bayesian Nonparametric Weighted Sampling Inference
- Improving the Effectiveness of Traceability Link Recovery using Hierarchical Bayesian Networks
- Warm Jupiters in TESS Full-Frame Images: A Catalog and Observed Eccentricity Distribution for Year 1
- Spherical Hamiltonian Monte Carlo for Constrained Target Distributions
- Hamiltonian Monte Carlo Without Detailed Balance
- Auxiliary-variable Exact Hamiltonian Monte Carlo Samplers for Binary Distributions
- Random projections for Bayesian regression
- Efficient Gradient-Based Inference through Transformations between Bayes Nets and Neural Nets
- Kinematic modelling of clusters with Gaia: the Death Throes of the Hyades
- Lagrangian Dynamical Monte Carlo
- Hierarchical Bayesian Modelling for Knowledge Transfer Across Engineering Fleets via Multitask Learning
- Approaching epidemiological dynamics of COVID-19 with physics-informed neural networks
- Capse.jl: efficient and auto-differentiable CMB power spectra emulation
- Markov Chain Monte Carlo-Based Machine Unlearning: Unlearning What Needs to be Forgotten
- Divide-and-Conquer with Sequential Monte Carlo
- Auto-Differentiable Spectrum Model for High-Dispersion Characterization of Exoplanets and Brown Dwarfs
- The evolution of the Milky Way's thin disc radial metallicity gradient with K2 asteroseismic ages
- On the accuracy and precision of correlation functions and field-level inference in cosmology
- RR Lyrae-based Distances for 39 Nearby Dwarf Galaxies Calibrated to Gaia eDR3
- TOI-836: A super-Earth and mini-Neptune transiting a nearby K-dwarf
- TESS spots a mini-neptune interior to a hot saturn in the TOI-2000 system
- Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale
- Adaptive Hamiltonian and Riemann Manifold Monte Carlo Samplers
- Bayesian aggregation of average data: An application in drug development
- sourceR: Classification and Source Attribution of Infectious Agents among Heterogeneous Populations
- Effective Bayesian Modeling of Groups of Related Count Time Series
- Intuitive Joint Priors for Bayesian Linear Multilevel Models: The R2D2M2 prior
- SIMD Parallel MCMC Sampling with Applications for Big-Data Bayesian Analytics
- AstroPhot: Fitting Everything Everywhere All at Once in Astronomical Images
- Identification of Basins of Attraction in the Local Universe
- Detecting False Rumors from Retweet Dynamics on Social Media
- On the evaluation of uncertainties for state estimation with the Kalman filter
- Impact of the Euro 2020 championship on the spread of COVID-19
- Measurement of the energy asymmetry in production at 13 TeV with the ATLAS experiment and interpretation in the SMEFT framework
- Some models are useful, but how do we know which ones? Towards a unified Bayesian model taxonomy
- Evaluating uncertainties in electrochemical impedance spectra of solid oxide fuel cells
- Conditional independence by typing
- Learning Regionalization using Accurate Spatial Cost Gradients within a Differentiable High-Resolution Hydrological Model: Application to the French Mediterranean Region
- Artificial mental phenomena: Psychophysics as a framework to detect perception biases in AI models
- Hurricanes and hashtags: Characterizing online collective attention for natural disasters
- High-dimensional Automated Radiation Therapy Treatment Planning via Bayesian Optimization
- COVID-19 detection using chest X-rays: is lung segmentation important for generalization?
- EEG-based performance-driven adaptive automated hazard alerting system in security surveillance support
- Multi-core parallel tempering Bayeslands for basin and landscape evolution
- Operational, gauge-free quantum tomography
- A tutorial on the Bayesian statistical approach to inverse problems
- Couplings for Andersen Dynamics
- The underlying radial acceleration relation
- Probabilistic Programming with Densities in SlicStan: Efficient, Flexible and Deterministic
- Confirmation of the Long-Period Planet Orbiting Gliese 411 and the Detection of a New Planet Candidate
- Precision Ephemerides for Gravitational-wave Searches -- IV: Corrected and refined ephemeris for Scorpius X-1
- Bayesian Non-parametric Quantile Process Regression and Estimation of Marginal Quantile Effects
- Infrared polarimetry of Mrk 231: Scattering off hot dust grains in the central core
- Two per cent measurement of from Cepheids alone
- TESS Giants Transiting Giants V -- Two hot Jupiters orbiting red-giant hosts
- Efficient and precise transit light curves for rapidly-rotating, oblate stars
- Cosmic Cartography: Bayesian reconstruction of the galaxy density informed by large-scale structure
- bssm: Bayesian Inference of Non-linear and Non-Gaussian State Space Models in R
- Bayesian Neural Networks
- Computing Bayes: From Then 'Til Now'
- Bayesian Optimisation for Active Monitoring of Air Pollution
- ProBO: Versatile Bayesian Optimization Using Any Probabilistic Programming Language
- Refining Epidemiological Forecasts with Simple Scoring Rules
- Modeling Longitudinal Dynamics of Comorbidities
- An approximate diffusion process for environmental stochasticity in infectious disease transmission modelling
- Modular Probabilistic Models via Algebraic Effects
- Informed reversible jump algorithms
- Contextual Motifs: Increasing the Utility of Motifs using Contextual Data
- Inferring sources of substandard and falsified products in pharmaceutical supply chains
- Core-periphery Models for Hypergraphs
- How Bayesian Should Bayesian Optimisation Be?
- Modelling variability power spectra of active galaxies from irregular time series
- Epidemic-guided deep learning for spatiotemporal forecasting of Tuberculosis outbreak
- Using a Bayesian-Inference Approach to Calibrating Models for Simulation in Robotics
- Exact Hamiltonian Monte Carlo for Truncated Multivariate Gaussians
- Deep Markov Chain Monte Carlo
- A Differentiable, End-to-End Forward Model for 21 cm Cosmology: Estimating the Foreground, Instrument, and Signal Joint Posterior
- A Probabilistic Simulator of Spatial Demand for Product Allocation
- Sequentially guided MCMC proposals for synthetic likelihoods and correlated synthetic likelihoods
- Confirming the Tidal Tails of the Young Open Cluster Blanco 1 with TESS Rotation Periods
- Testing the local supervoid solution to the Hubble tension with direct distance tracers
- Neural parameter calibration and uncertainty quantification for epidemic forecasting
- Differentially Private Bayesian Inference for Generalized Linear Models
- Leveraging Human Feedback to Evolve and Discover Novel Emergent Behaviors in Robot Swarms
- Independent finite approximations for Bayesian nonparametric inference
- Dichroic polarization at mid-infrared wavelengths: a Bayesian approach
- A reversible infinite HMM using normalised random measures
- Clustering properties of the CatWISE2020 quasar catalogue and their impact on the cosmic dipole anomaly
- Bispectrum constraints on Primordial non-Gaussianities with the eBOSS DR16 quasars
- Hawai'i Supernova Flows: A Peculiar Velocity Survey Using Over a Thousand Supernovae in the Near-Infrared
- Sampling algorithms in statistical physics: a guide for statistics and machine learning
- JAXNS: a high-performance nested sampling package based on JAX
- Bayesian Hierarchical Models for Quantitative Estimates for Performance metrics applied to Saddle Search Algorithms
- The Velocity Field Olympics: Assessing velocity field reconstructions with direct distance tracers
- Exploring HOD-dependent systematics for the DESI 2024 Full-Shape galaxy clustering analysis
- Self-Tuning Hamiltonian Monte Carlo for Accelerated Sampling
- Bayesian modeling of insurance claims for hail damage
- Bayesian posterior approximation with stochastic ensembles
- Bayesian propensity score matching in automotive embedded software engineering
- Instantaneous turbulent kinetic energy modelling based on Lagrangian stochastic approach in CFD and application to wind energy
- Bayesian Variable Selection for Non-Gaussian Responses: A Marginally Calibrated Copula Approach
- Tempering by Subsampling
- Split Hamiltonian Monte Carlo
- Sparse Uncertainty Representation in Deep Learning with Inducing Weights
- No evidence for local anisotropy from Tully--Fisher or supernova distances
- DeepSSM: an emulator of gravitational wave spectra from sound waves during cosmological first-order phase transitions
- Bayesian Functional Emulation of CO2 Emissions on Future Climate Change Scenarios
- Recycling intermediate steps to improve Hamiltonian Monte Carlo
- Delayed rejection Hamiltonian Monte Carlo for sampling multiscale distributions
- Personalized Product Assortment with Real-time 3D Perception and Bayesian Payoff Estimation
- Constraining the Rotation Profile in a Low-Luminosity Subgiant with a Surface Rotation Measurement
- Hierarchical Bayesian estimation of motor-evoked potential recruitment curves yields accurate and robust estimates
- Modeling racial/ethnic differences in COVID-19 incidence with covariates subject to non-random missingness
- Probabilistic Programming and PyMC3
- A Geometric View of Posterior Approximation
- Constraining the primordial power spectrum using a differentiable likelihood
- Statistical Models for Repeated Categorical Ratings: The R Package rater
- On decision-theoretic model assessment for structural deterioration monitoring
- Generalized Decomposition Priors on R2
- Gaussian distributional structural equation models: A framework for modeling latent heteroscedasticity
- Interpretable Stochastic Block Influence Model: measuring social influence among homophilous communities
- A Search for Planetary Metastable Helium Absorption in the V1298 Tau System
- Unadjusted Hamiltonian MCMC with Stratified Monte Carlo Time Integration
- Stress Resultant-Based Approach to Mass Assumption-Free Bayesian Model Updating of Frame Structures
- Gravitational wave energy spectral density properties from BPASS Galactic binary population in the Milky Way galaxy
- Hierarchical Bayesian inference for uncertainty quantification of thermal grease rheology
- Trans-dimensional Hamiltonian model selection and parameter estimation from sparse, noisy data
- Precomputing Strategy for Hamiltonian Monte Carlo Method Based on Regularity in Parameter Space
- Forward-modelling Milky Way Cepheids: selection effects and physical priors in the Gaia-HST calibration
- Efficient computation of high-dimensional penalized piecewise constant hazard random effects models
- Monte Carlo guided Diffusion for Bayesian linear inverse problems
- Discrepancies in Epidemiological Modeling of Aggregated Heterogeneous Data
- TOI-3362b: A Proto-Hot Jupiter Undergoing High-Eccentricity Tidal Migration
- TyXe: Pyro-based Bayesian neural nets for Pytorch
- Gibbs Sampling using Anti-correlation Gaussian Data Augmentation, with Applications to L1-ball-type Models
- A Compartment Model of Human Mobility and Early Covid-19 Dynamics in NYC
- Fixed-kinetic Neural Hamiltonian Flows for enhanced interpretability and reduced complexity
- Neural network emulator to constrain the high- IGM thermal state from Lyman- forest flux auto-correlation function
- from peculiar velocities: agreement with Planck for Tully--Fisher and supernovae, tension for the fundamental plane
- Parameterizations for Gradient-based Markov Chain Monte Carlo on the Stiefel Manifold: A Comparative Study
- Probabilistic Programs with Stochastic Conditioning
- The Sequential Monte Carlo goes NUTS: Boosting Gravitational-Wave Inference
- Merger or Not: Accounting for Human Biases in Identifying Galactic Merger Signatures
- Approximate Gibbs Sampler for Efficient Inference of Hierarchical Bayesian Models for Grouped Count Data