When can a posterior predictive check identify the learning rate? Exact degeneracy in Gaussian models and implications for Generalised Bayesian Inerence

Abstract

Generalised Bayesian inference tempers the likelihood by a learning rate $\hat{I}\cdot$ to mitigate model misspecification, and the choice of $\hat{I}\cdot$ is consequential. Zafar and Nicholls (2024) proposed selecting $\hat{I}\cdot$ by a posterior predictive check (PPC): one chooses the smallest $\hat{I}\cdot$ at which a log-likelihood PPC $p$-value is not rejected. An exact, finite-sample analysis of this selector on the Gaussian linear model is given. With known variance and a flat prior, the PPC $p$-value equals $P(\ddot{I}{‡}_n^2>\mathrm{RSS}/\ddot{I}{ƒ}_0^2)$ for every $\hat{I}\cdot$, so the selector is $\hat{I}\cdot$-invariant; under variance misspecification it is two-sided non-identifying. With unknown variance and the reference prior, the $p$-value depends only on $(n,d,\hat{I}\cdot )$ and not on the realised data or the data-generating process. Consequently the selector's output is fixed before any data are seen, typically collapsing to the smallest grid value, which over-tempers and inflates predictive intervals relative to held-out selection. The phenomenon is a pivotality property specific to the Gaussian scale--location family and the reference prior; it disappears under informative priors. These results delineate the selector's scope, identify a canonical class on which it cannot identify the learning rate, and motivate a cheap, data-free pre-screening diagnostic.

6 pages, 4 figures