Rethinking the Effective Sample Size

Abstract

The effective sample size (ESS) is widely used in sample-based simulation methods for assessing the quality of a Monte Carlo approximation of a given distribution and of related integrals. In this paper, we revisit the approximation of the ESS in the specific context of importance sampling (IS). The derivation of this approximation, that we will denote as $\widehat{\text{ESS}}$, is partially available in Kong (1992). This approximation has been widely used in the last 25 years due to its simplicity as a practical rule of thumb in a wide variety of importance sampling methods. However, we show that the multiple assumptions and approximations in the derivation of $\widehat{\text{ESS}}$, makes it difficult to be considered even as a reasonable approximation of the ESS. We extend the discussion of the $\widehat{\text{ESS}}$ in the multiple importance sampling (MIS) setting, we display numerical examples, and we discuss several avenues for developing alternative metrics. This paper does not cover the use of ESS for MCMC algorithms.

References in corpus (4)

• Adaptive Importance Sampling in General Mixture Classes
• Importance Gaussian Quadrature
• Compressed Monte Carlo with application in particle filtering
• Multiple Importance Sampling for Efficient Symbol Error Rate Estimation

Cited by in corpus (29)

• Massively parallel Bayesian inference for transient gravitational-wave astronomy
• Phase equilibrium of water with hexagonal and cubic ice using the SCAN functional
• Importance Gaussian Quadrature
• Implicitly Adaptive Importance Sampling
• Fortifying gravitational-wave tests of general relativity against astrophysical assumptions
• The Return of GOLUM: Improving Distributed Joint Parameter Estimation for Strongly-Lensed Gravitational Waves
• Iterative importance sampling with Markov chain Monte Carlo sampling in robust Bayesian analysis
• MCMC-driven importance samplers
• Rare Events via Cross-Entropy Population Monte Carlo
• A comparison of Bayesian sampling algorithms for high-dimensional particle physics and cosmology applications
• Rapid inference and comparison of gravitational-wave population models with neural variational posteriors
• Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weighting
• Effective Sample Size, Dimensionality, and Generalization in Covariate Shift Adaptation
• Online Particle Smoothing with Application to Map-matching
• Multifidelity Approximate Bayesian Computation with Sequential Monte Carlo Parameter Sampling
• The Impact of Astrophysical Priors on Parameter Inference for GW230529
• Efron-Stein PAC-Bayesian Inequalities
• Nearly Consistent Finite Particle Estimates in Streaming Importance Sampling
• An index of effective number of variables for uncertainty and reliability analysis in model selection problems
• A primer on inference and prediction with epidemic renewal models and sequential Monte Carlo
• Overlapping signals in next-generation gravitational wave observatories: A recipe for selecting the best parameter estimation technique
• Flow Annealed Importance Sampling Bootstrap meets Differentiable Particle Physics
• Meta Off-Policy Estimation
• Advances in Importance Sampling
• Doubly Adaptive Importance Sampling
• Parallelizable Feynman-Kac Models for Universal Probabilistic Programming
• Effective sample size approximations as entropy measures
• Efficient Nested Simulation Experiment Design via the Likelihood Ratio Method
• Inferring planet occurrence rates from radial velocities