Bessel-Debiased Pseudo-Marginal MCMC for Generalised Bayesian Inference

Abstract

Generalised Bayesian inference uses weights of the form $\exp \{-{\beta }_n{\ell }_n(\theta )\}$ even when the loss is only estimated. Exponentiating an unbiased loss estimate changes the target, and when ${\beta }_n\asymp n$ an ordinary Monte Carlo loss estimate with variance of order $M^{-1}$ requires a per-proposal budget $M$ of order $n^2$ to keep the leading log-weight variance bounded. We introduce Sign-Corrected Bessel Debiasing (SCBD), a signed pseudo-marginal method based on independent block estimates of the loss, and study its ordinary-MC and independently randomised quasi-Monte Carlo (RQMC) implementations. Under an i.i.d. Gaussian block model, a Bessel factor constructed from the block sample variance exactly removes the Gaussian exponential inflation despite the variance being unknown. For general non-Gaussian finite blocks, the method targets a posterior differing from the intended posterior by a parameter-dependent multiplicative factor. Under regularity conditions, the uncorrected and corrected ordinary-MC targets have total-variation errors of orders ${\beta }_n^2/M_n$ and ${\beta }_n^3/M_n^2$. If an RQMC block estimator has variance $\mathcal{O}\{B^{-\alpha }(\log B{)}^{d-1}\}$, the corresponding errors are of orders ${\delta }_{n,M_n}^{\mathrm{RQ}}$ and $({\delta }_{n,M_n}^{\mathrm{RQ}}{)}^{3/2}$, where ${\delta }_{n,M_n}^{\mathrm{RQ}}={\beta }_n^2{M}_n^{-\alpha }{\log (2+M_n)}^{d-1}$. The same variance rate gives a sufficient budget of order $n^{2/\alpha }$, up to logarithmic factors, for bounded leading log-weight variance when ${\beta }_n\asymp n$. The numerical examples show that favourable RQMC representations can inherit this budget scaling and that variance reduction and Bessel correction are complementary. Compared to existing exact corrections, Bessel debiasing is essentially "for free". It is generic, easy to code and supported by theory.

55 pages, 11 figures