Occam factors and model-independent Bayesian learning of continuous distributions

Abstract

Learning of a smooth but nonparametric probability density can be regularized using methods of Quantum Field Theory. We implement a field theoretic prior numerically, test its efficacy, and show that the data and the phase space factors arising from the integration over the model space determine the free parameter of the theory ("smoothness scale") self-consistently. This persists even for distributions that are atypical in the prior and is a step towards a model-independent theory for learning continuous distributions. Finally, we point out that a wrong parameterization of a model family may sometimes be advantageous for small data sets.

publication revisions: extended introduction, new references, other minor corrections; 6 pages, 6 figures, revtex

References in corpus (4)

• Field Theories for Learning Probability Distributions
• The Analysis of Data from Continuous Probability Distributions
• Reparametrization invariant statistical inference and gravity
• Field Theoretical Analysis of On-line Learning of Probability Distributions

Cited by in corpus (9)

• Inference of entropies of discrete random variables with unknown cardinalities
• Physical limit to concentration sensing in a changing environment
• Density estimation on small datasets
• Rapid and deterministic estimation of probability densities using scale-free field theories
• Stereotypical escape behavior in Caenorhabditis elegans allows quantification of nociceptive stimuli levels
• Unification of field theory and maximum entropy methods for learning probability densities
• Bayesian field theoretic reconstruction of bond potential and bond mobility in single molecule force spectroscopy
• Neural coding of a natural stimulus ensemble: Uncovering information at sub-millisecond resolution
• Fluctuation-dissipation theorem and models of learning