Fast MCMC sampling for Markov jump processes and extensions

Abstract

Markov jump processes (or continuous-time Markov chains) are a simple and important class of continuous-time dynamical systems. In this paper, we tackle the problem of simulating from the posterior distribution over paths in these models, given partial and noisy observations. Our approach is an auxiliary variable Gibbs sampler, and is based on the idea of uniformization. This sets up a Markov chain over paths by alternately sampling a finite set of virtual jump times given the current path and then sampling a new path given the set of extant and virtual jump times using a standard hidden Markov model forward filtering-backward sampling algorithm. Our method is exact and does not involve approximations like time-discretization. We demonstrate how our sampler extends naturally to MJP-based models like Markov-modulated Poisson processes and continuous-time Bayesian networks and show significant computational benefits over state-of-the-art MCMC samplers for these models.

Accepted at the Journal of Machine Learning Research (JMLR)

References in corpus (3)

• Continuous Time Bayesian Networks
• Expectation Propagation for Continuous Time Bayesian Networks
• Mean Field Variational Approximation for Continuous-Time Bayesian Networks

Cited by in corpus (26)

• Increasing the Interpretability of Recurrent Neural Networks Using Hidden Markov Models
• Path Weight Sampling: Exact Monte Carlo Computation of the Mutual Information between Stochastic Trajectories
• Bayesian Inference for Hybrid Discrete-Continuous Stochastic Kinetic Models
• Markov-Modulated Hawkes Processes for Sporadic and Bursty Event Occurrences
• Avoiding matrix exponentials for large transition rate matrices
• Uniformisation techniques for stochastic simulation of chemical reaction networks
• Constrained Approximation of Effective Generators for Multiscale Stochastic Reaction Networks and Application to Conditioned Path Sampling
• Efficient parameter sampling for Markov jump processes
• Optimizing city-scale traffic through modeling observations of vehicle movements
• Fast and Flexible Temporal Point Processes with Triangular Maps
• Auxiliary Variable Markov Chain Monte Carlo for Spatial Survival and Geostatistical Models
• Metropolis-type algorithms for Continuous Time Bayesian Networks
• Scalable Bayesian Inference for Population Markov Jump Processes
• Particle Gibbs algorithms for Markov jump processes
• Belief dynamics extraction
• Active Learning of Continuous-time Bayesian Networks through Interventions
• An Exact Auxiliary Variable Gibbs Sampler for a Class of Diffusions
• Integrated Continuous-time Hidden Markov Models
• Auxiliary Variables for Bayesian Inference in Multi-Class Queueing Networks
• Bayesian inference for the Markov-modulated Poisson process with an outcome process
• Efficient sampling of conditioned Markov jump processes
• Deep Neyman-Scott Processes
• Geometric ergodicity of Rao and Teh's algorithm for Markov jump processes
• Variational inequalities and mean-field approximations for partially observed systems of queueing networks
• On Bayesian inferential tasks with infinite-state jump processes: efficient data augmentation
• Scalable Bayesian Multiple Changepoint Detection via Auxiliary Uniformization