Bayesian analysis for reversible Markov chains

Abstract

We introduce a natural conjugate prior for the transition matrix of a reversible Markov chain. This allows estimation and testing. The prior arises from random walk with reinforcement in the same way the Dirichlet prior arises from Pólya's urn. We give closed form normalizing constants, a simple method of simulation from the posterior and a characterization along the lines of W. E. Johnson's characterization of the Dirichlet prior.

Published at http://dx.doi.org/10.1214/009053606000000290 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Cited by in corpus (21)

• A survey of random processes with reinforcement
• An asymptotic result for Brownian polymers
• Random walks in random Dirichlet environment are transient in dimension $d\ge 3$
• The Dirichlet Markov Ensemble
• Bayesian analysis of variable-order, reversible Markov chains
• On the transience of processes defined on Galton--Watson trees
• Linearly edge-reinforced random walks
• On a class of random walks with reinforced memory
• Edge-reinforced random walk on a ladder
• Information Geometry of Reversible Markov Chains
• Properties of Uniform Doubly Stochastic Matrices
• Asymptotic behavior of edge-reinforced random walks
• Algebraic Bayesian analysis of contingency tables with possibly zero-probability cells
• Bayesian nonparametric analysis of reversible Markov chains
• Edge-reinforced random walk, Vertex-Reinforced Jump Process and the supersymmetric hyperbolic sigma model
• Invariance principles for random walks in random environment on trees
• A note on recurrence of the Vertex reinforced jump process and fractional moments localization
• Another Conversation with Persi Diaconis
• Spin systems with hyperbolic symmetry: a survey
• Reversible Markov chain estimation using convex-concave programming
• Scaling limit of linearly edge-reinforced random walks on critical Galton-Watson trees