Dynamic nested sampling: an improved algorithm for parameter estimation and evidence calculation

Abstract

We introduce dynamic nested sampling: a generalisation of the nested sampling algorithm in which the number of "live points" varies to allocate samples more efficiently. In empirical tests the new method significantly improves calculation accuracy compared to standard nested sampling with the same number of samples; this increase in accuracy is equivalent to speeding up the computation by factors of up to ~72 for parameter estimation and ~7 for evidence calculations. We also show that the accuracy of both parameter estimation and evidence calculations can be improved simultaneously. In addition, unlike in standard nested sampling, more accurate results can be obtained by continuing the calculation for longer. Popular standard nested sampling implementations can be easily adapted to perform dynamic nested sampling, and several dynamic nested sampling software packages are now publicly available.

14 pages + appendix, 16 figures. Added signal reconstruction numerical example using dyPolyChord. Moderate updates to text and appendices

References in corpus (21)

• MultiNest: an efficient and robust Bayesian inference tool for cosmology and particle physics
• Multimodal nested sampling: an efficient and robust alternative to MCMC methods for astronomical data analysis
• Dark Energy Survey Year 1 Results: Cosmological Constraints from Galaxy Clustering and Weak Lensing
• Importance Nested Sampling and the MultiNest Algorithm
• PolyChord: nested sampling for cosmology
• PolyChord: next-generation nested sampling
• MOSFiT: Modular Open-Source Fitter for Transients
• Properties of Nested Sampling
• Bayesian model selection without evidences: application to the dark energy equation-of-state
• Comparison of sampling techniques for Bayesian parameter estimation
• fgivenx: A Python package for functional posterior plotting
• Sampling Errors in Nested Sampling Parameter Estimation
• Recursive Pathways to Marginal Likelihood Estimation with Prior-Sensitivity Analysis
• nestcheck: diagnostic tests for nested sampling calculations
• Improving power posterior estimation of statistical evidence
• On statistical uncertainty in nested sampling
• Superposition Enhanced Nested Sampling
• Nested Sampling with Constrained Hamiltonian Monte Carlo
• Bayesian sparse reconstruction: a brute-force approach to astronomical imaging and machine learning
• Constraining the Stellar Mass Function from the Deficiency of Tidal Disruption Flares in the Nuclei of Massive Galaxies
• DNest4: Diffusive Nested Sampling in C++ and Python

Cited by in corpus (178)

• dynesty: A Dynamic Nested Sampling Package for Estimating Bayesian Posteriors and Evidences
• Importance Nested Sampling and the MultiNest Algorithm
• Stellar Population Inference with Prospector
• Juliet: a versatile modelling tool for transiting and non-transiting exoplanetary systems
• STRIDES: a 3.9 per cent measurement of the Hubble constant from the strong lens system DES J0408-5354
• 3-OGC: Catalog of gravitational waves from compact-binary mergers
• Six transiting planets and a chain of Laplace resonances in TOI-178
• TauREx III: A fast, dynamic and extendable framework for retrievals
• Nested sampling for physical scientists
• Life beyond 30: probing the -20<M_UV<-17 luminosity function at 8<z<13 with the NIRCam parallel field of the MIRI Deep Survey
• The Science Case for LIGO-India
• ARIADNE: Measuring accurate and precise stellar parameters through SED fitting
• Measuring the dark matter environments of black hole binaries with gravitational waves
• Nested Sampling Methods
• Occurrence rate of hot Jupiters around early-type M dwarfs based on TESS data
• Evidence for an intermediate-mass black hole from a gravitationally lensed gamma-ray burst
• Search for Axionlike-Particle-Induced Prompt Gamma-Ray Emission from Extragalactic Core-Collapse Supernovae with the Fermi Large Area Telescope
• The Return of the Templates: Revisiting the Galactic Center Excess with Multi-Messenger Observations
• Aurora: A Generalised Retrieval Framework for Exoplanetary Transmission Spectra
• Present-day mass-metallicity relation for galaxies using a new electron-temperature method
• A dynamical measure of the black hole mass in a quasar 11 billion years ago
• Dispersive Analysis of $B\to K^{(\ast )}$ and $B_s\to \varphi$ Form Factors
• Testing the Cosmological Principle with CatWISE Quasars: A Bayesian Analysis of the Number-Count Dipole
• Evolution of the Mass-Metallicity Relation from Redshift $z\approx 8$ to the Local Universe
• JWST transmission spectroscopy of HD 209458b: a super-solar metallicity, a very low C/O, and no evidence of CH4, HCN, or C2H2
• Towards inference of overlapping gravitational wave signals
• Fast and Credible Likelihood-Free Cosmology with Truncated Marginal Neural Ratio Estimation
• TOI-1452 b: SPIRou and TESS reveal a super-Earth in a temperate orbit transiting an M4 dwarf
• Simple-graduated dark energy and spatial curvature
• Detection of the tidal deformation of WASP-103b at $3\sigma$ with CHEOPS
• Measuring dark matter spikes around primordial black holes with Einstein Telescope and Cosmic Explorer
• Spi-OPS: Spitzer and CHEOPS confirm the near-polar orbit of MASCARA-1 b and reveal a hint of dayside reflection
• The SAMI Galaxy Survey: stellar population and structural trends across the Fundamental Plane
• A detailed analysis of the Gl 486 planetary system
• Importance nested sampling with normalising flows
• The GAPS Programme with HARPS-N at TNG. XXXVII. A precise density measurement of the young ultra-short period planet TOI-1807 b
• Parametrized tests of post-Newtonian theory using principal component analysis
• The Magellan-TESS Survey I: Survey Description and Mid-Survey Results
• TOI-530b: A giant planet transiting an M dwarf detected by TESS
• Caught in the Act: Core-powered Mass-loss Predictions for Observing Atmospheric Escape
• Constraints on warm dark matter from UV luminosity functions of high-z galaxies with Bayesian model comparison
• Performance forecasts for the primordial gravitational wave detection pipelines for AliCPT-1
• Toward the unambiguous identification of supermassive binary black holes through Bayesian inference
• The stellar populations of quiescent ultra-diffuse galaxies from optical to mid-infrared spectral energy distribution fitting
• Bayesian Inference of the Dense Matter Equation of State built upon Covariant Density Functionals
• First semi-empirical test of the white dwarf mass-radius relationship using a single white dwarf via astrometric microlensing
• Where intermediate-mass black holes could hide in the Galactic Centre: A full parameter study with the S2 orbit
• Betelgeuse fainter in the sub-millimetre too: an analysis of JCMT and APEX monitoring during the recent optical minimum
• The geometric albedo of the hot Jupiter HD 189733b measured with CHEOPS
• VARAHA: A Fast Non-Markovian sampler for estimating Gravitational-Wave posteriors
• GRB 200716C: Evidence for a Short Burst Being Lensed
• Direct Constraints on the Extremely Metal-Poor Massive Stars Underlying Nebular C IV Emission from Ultra-Deep HST/COS Ultraviolet Spectroscopy
• Towards a framework for testing general relativity with extreme-mass-ratio-inspiral observations
• Still at Odds with Conventional Galaxy Evolution: The Star Formation History of Ultra-Diffuse Galaxy Dragonfly 44
• Dark matter profiles of SPARC galaxies: a challenge to fuzzy dark matter
• An ALMA+ACA measurement of the shock in the Bullet Cluster
• Constraining spontaneous black hole scalarization in scalar-tensor-Gauss-Bonnet theories with current gravitational-wave data
• DESI Dark Energy Time Evolution is Recovered by Cosmologically Coupled Black Holes
• Ogle-2018-blg-0677lb: A super earth near the galactic bulge
• Planets larger than Neptune have elevated eccentricities
• The Hubble PanCET Program: A Metal-rich Atmosphere for the Inflated Hot Jupiter HAT-P-41b
• Fitting the light curves of Sagittarius A* with a hot-spot model
• Multimass modelling of Milky Way globular clusters -- I. Implications on their stellar initial mass function above 1 M${}_{\odot }$
• The cosmic build-up of dust and metals. Accurate abundances from GRB-selected star-forming galaxies at $1.7<z<6.3$
• Fast and robust Bayesian Inference using Gaussian Processes with GPry
• Exploring over 700 massive quiescent galaxies at z = 2-7: Demographics and stellar mass functions
• The Aemulus Project VI: Emulation of beyond-standard galaxy clustering statistics to improve cosmological constraints
• TESS discovery of a sub-Neptune orbiting a mid-M dwarf TOI-2136
• Precision constraints on the neutron star equation of state with third-generation gravitational-wave observatories
• Evidence of wind signatures in the gas velocity profiles of Red Geysers
• Stellar Populations in the Central 0.5 pc of Our Galaxy III: The Dynamical Sub-structures
• Bayesian Model Selection with Future 21cm Observations of The Epoch of Reionisation
• Probing the large scale structure using gravitational-wave observations of binary black holes
• A candidate relativistic tidal disruption event at 340 Mpc
• A microlensing search of 700 million VVV light curves
• Signatures of inflowing gas in red geyser galaxies hosting radio-AGN
• A comprehensive revisit of select Galileo/NIMS observations of Europa
• Does the speed of gravitational waves depend on the source velocity?
• Bayesian sparse reconstruction: a brute-force approach to astronomical imaging and machine learning
• VVV-WIT-08: the giant star that blinked
• Search for eccentric NSBH and BNS mergers in the third observing run of Advanced LIGO and Virgo
• Toward a complete description of $b\to u{\ell }^{-}\bar{\nu }$ decays within the Weak Effective Theory
• Dark lenses through the dust: parallax microlensing events in the VVV
• GRB 220426A: A Thermal Radiation-Dominated Gamma-Ray Burst
• Nested sampling with plateaus
• New constraints on the central mass contents of Omega Centauri from combined stellar kinematics and pulsar timing
• The nascent milliquasar VT J154843.06+220812.6: tidal disruption event or extreme accretion-state change?
• Bayesian analysis of Juno/JIRAM's NIR observations of Europa
• Bayesian evidence estimation from posterior samples with normalizing flows
• A systematic validation of hot Neptunes in TESS data
• A dense mini-Neptune orbiting the bright young star HD 18599
• Optical polarization and spectral properties of the H-poor superluminous supernovae SN 2021bnw and SN 2021fpl
• Allesfitter: Flexible Star and Exoplanet Inference From Photometry and Radial Velocity
• Post maximum light and late time optical imaging polarimetry of type I superluminous supernova 2020znr
• Constraining the Stellar Mass Function from the Deficiency of Tidal Disruption Flares in the Nuclei of Massive Galaxies
• Uncovering the Invisible: A Study of Gaia18ajz, a Candidate Black Hole Revealed by Microlensing
• Bayesian fitting of multi-Gaussian expansion models to galaxy images
• A 16 Hour Transit of Kepler-167 e Observed by the Ground-based Unistellar Telescope Network
• CHEOPS and TESS view of the ultra-short period super-Earth TOI-561 b
• A comparison of Bayesian sampling algorithms for high-dimensional particle physics and cosmology applications
• Planetary companions orbiting the M dwarfs GJ 724 and GJ 3988. A CARMENES and IRD collaboration
• Exploring phase space with Nested Sampling
• Hiding in plain sight: observing planet-starspot crossings with the James Webb Space Telescope
• Discriminating among cosmological models by data-driven methods
• Realizing the potential of deep neural network for analyzing neutron star observables and dense matter equation of state
• Revisiting the HD 21749 Planetary System with Stellar Activity Modeling
• Constraining $|V_{cs}|$ and physics beyond the Standard Model from exclusive (semi)leptonic charm decays
• Mapping the Skies of Ultracool Worlds: Detecting Storms and Spots with Extremely Large Telescopes
• GalaPy, the highly optimised C++/Python spectral modelling tool for galaxies -- I. Library presentation and photometric fitting
• BASS XLVII: 22 GHz Radio Atlas of Swift-BAT Selected AGN
• Detecting Baryon Acoustic Oscillations with third generation gravitational wave observatories
• Uniform Modelling of the Stellar Density of Thirteen Tidal Streams within the Galactic Halo
• HD 183579b: A Warm Sub-Neptune Transiting a Solar Twin Detected by TESS
• The proper motion of stars in dwarf galaxies: distinguishing central density cusps from cores
• Differential Interferometric Signatures of Close Binaries of Supermassive Black Holes in Active Galactic Nuclei: II. Merged Broad Line Regions
• Flow Matching for Atmospheric Retrieval of Exoplanets: Where Reliability meets Adaptive Noise Levels
• Dark skies of the slightly eccentric WASP-18 b from its optical-to-infrared dayside emission
• The Frequency and Mass-Ratio Distribution of Binaries in Clusters I: Description of the method and application to M67
• Impact of higher harmonics of gravitational radiation on the population inference of binary black holes
• TIC~257060897b: an inflated, low-density, hot-Jupiter transiting a rapidly evolving subgiant star
• Exploiting the transit timing capabilities of Ariel
• Accelerating Tests of General Relativity with Gravitational-Wave Signals using Hybrid Sampling
• Constraints on Bianchi-I type universe with SH0ES anchored Pantheon+ SNIa data
• Two neighbours of the ultra-short-period Earth-sized planet K2-157 b in the warm Neptunian savanna
• SN 2019odp: A Massive Oxygen-Rich Type Ib Supernova
• Low-Frequency Turnover Star Forming Galaxies I: Radio Continuum Observations and Global Properties
• NGTS-21b: An Inflated Super-Jupiter Orbiting a Metal-poor K dwarf
• RadioSED I: Bayesian inference of radio SEDs from inhomogeneous surveys
• The SPACE Program. I. The featureless spectrum of HD 86226 c challenges sub-Neptune atmosphere trends
• The soft X-ray transient EP241021a: A cosmic explosion with a complex off-axis jet and cocoon from a massive progenitor
• Fast and accurate parameter estimation of high-redshift sources with the Einstein Telescope
• Modeling Star Formation Histories of Changing-Look AGN Host Galaxies with Prospector
• Fast and faithful interpolation of numerical relativity surrogate waveforms using meshfree approximation
• The GAPS Programme at TNG. LXV. Precise density measurement of TOI-1430 b, a young planet with an evaporating atmosphere
• A Stringent Upper Limit on Dark Matter Self-Interaction Cross Section from Cluster Strong Lensing
• A probabilistic approach to phase calibration: I. Effects of source structure on fringe-fitting
• GRB 201104A: A "Repetitive" Short Gamma-Ray Burst?
• Population synthesis and parameter estimation of neutron stars with continuous gravitational waves and third-generation detectors
• Nested Sampling And Likelihood Plateaus
• Detectability of Gravitational-Wave Memory with LISA: A Bayesian Approach
• Tidally Heated Sub-Neptunes, Refined Planetary Compositions, and Confirmation of a Third Planet in the TOI-1266 System
• Precise radial velocities of giant stars -- XVII. Distinguishing planets from intrinsically induced radial velocity signals in evolved stars
• Impact on $L$-observables of a new combined analysis of $B_{d,s}\to K^{(\ast )}$ form factors
• Kernel-, mean- and noise-marginalised Gaussian processes for exoplanet transits and $H_0$ inference
• Gravitational-wave model for neutron star merger remnants with supervised learning
• Evidence for a Nonzero Eccentricity Superpuff Exoplanet WASP-107 b Using JWST Occultation Observation
• A Moderate Albedo from Reflecting Aerosols on the Dayside of WASP-80 b Revealed by JWST/NIRISS Eclipse Spectroscopy
• Arbitrary Marginal Neural Ratio Estimation for Simulation-based Inference
• A practical Bayesian method for gravitational-wave ringdown analysis with multiple modes
• RedDots: Planetary masses in the GJ1061 system from planet-planet interaction
• AGN in the ULIRG HE 0435-5304
• Bayesian Analysis of the ${}^{86}$Sr$(\alpha ,\alpha )$ Reaction to Constrain the ${}^{86}$Sr$(\alpha ,n)$ Cross Section at Astrophysical Energies
• Inferring Evidence from Nested Sampling Data via Information Field Theory
• TESS phase curve of ultra-hot Jupiter WASP-189 b
• Closing the Evidence Gap: reddemcee, a Fast Adaptive Parallel Tempering Sampler
• Improved Identification of Strongly Lensed Gravitational Waves with Host Galaxy Locations
• Optical ray tracing of echelle spectrographs applied to the wavelength solution for precise radial velocities
• Search for giant planets in M67 V: a warm Jupiter orbiting the turn-off star S1429
• Statistical biases in parametrized searches for gravitational-wave polarizations
• The dark and featureless surface of rocky exoplanet LHS 3844 b from JWST mid-infrared spectroscopy
• Non-parametric and continuous extraction of amplitudes in electroweak penguin decays
• Probing the disk-jet coupling in M87
• Inferring binary black holes stellar progenitors with gravitational wave sources
• Inferring planet occurrence rates from radial velocities
• TOI-333b: A Neptune Desert planet around a F7V star
• Dynamical analysis of the 5:1 mean-motion resonance of the HD 202206 system
• MINDS survey of silicates in T Tauri disks: Correlation between dust and gas
• Short gamma-ray burst progenitors have short delay times
• Constraining the population of dark matter halo shapes using hierarchical inference with extragalactic stellar streams
• ExoEcho: Exoplanet Ephemerides Change Observations. III. A JWST Survey for Transit Timing Variations of 37 Exoplanets
• Two Rings and a Marginally Resolved, 5 AU, Disk Around LkCa 15 Identified Via Near Infrared Sparse Aperture Masking Interferometry
• Revisiting the secondary eclipses of KELT-1b using TESS observations
• Black-box density function estimation using recursive partitioning
• Global Determination of $|V_{us}|$
• Towards a Field Based Bayesian Evidence Inference from Nested Sampling Data
• Mapping the 3D Kinematical Structure of the Gas Disk of HD 169142
• The Progenitor Systems of Classical Novae in M31
• Bayesian vs Frequentist: Comparing Bayesian model selection with a frequentist approach using the iterative smoothing method