Group Importance Sampling for Particle Filtering and MCMC

Abstract

Bayesian methods and their implementations by means of sophisticated Monte Carlo techniques have become very popular in signal processing over the last years. Importance Sampling (IS) is a well-known Monte Carlo technique that approximates integrals involving a posterior distribution by means of weighted samples. In this work, we study the assignation of a single weighted sample which compresses the information contained in a population of weighted samples. Part of the theory that we present as Group Importance Sampling (GIS) has been employed implicitly in different works in the literature. The provided analysis yields several theoretical and practical consequences. For instance, we discuss the application of GIS into the Sequential Importance Resampling framework and show that Independent Multiple Try Metropolis schemes can be interpreted as a standard Metropolis-Hastings algorithm, following the GIS approach. We also introduce two novel Markov Chain Monte Carlo (MCMC) techniques based on GIS. The first one, named Group Metropolis Sampling method, produces a Markov chain of sets of weighted samples. All these sets are then employed for obtaining a unique global estimator. The second one is the Distributed Particle Metropolis-Hastings technique, where different parallel particle filters are jointly used to drive an MCMC algorithm. Different resampled trajectories are compared and then tested with a proper acceptance probability. The novel schemes are tested in different numerical experiments such as learning the hyperparameters of Gaussian Processes, two localization problems in a wireless sensor network (with synthetic and real data) and the tracking of vegetation parameters given satellite observations, where they are compared with several benchmark Monte Carlo techniques. Three illustrative Matlab demos are also provided.

To appear in Digital Signal Processing. Related Matlab demos are provided at https://github.com/lukafree/GIS.git

References in corpus (6)

• Effective Sample Size for Importance Sampling based on discrepancy measures
• Generalized Multiple Importance Sampling
• Cooperative Parallel Particle Filters for online model selection and applications to Urban Mobility
• A Review of Multiple Try MCMC algorithms for Signal Processing
• Efficient Multiple Importance Sampling Estimators
• Heretical Multiple Importance Sampling

Cited by in corpus (8)

• A Review of Multiple Try MCMC algorithms for Signal Processing
• Marginal likelihood computation for model selection and hypothesis testing: an extensive review
• Rethinking the Effective Sample Size
• Kernel Methods and their derivatives: Concept and perspectives for the Earth system sciences
• Robust Covariance Adaptation in Adaptive Importance Sampling
• Cluster Prediction for Opinion Dynamics from Partial Observations
• Deep Importance Sampling based on Regression for Model Inversion and Emulation
• Advances in Importance Sampling