Minimum Description Length Revisited

Abstract

This is an up-to-date introduction to and overview of the Minimum Description Length (MDL) Principle, a theory of inductive inference that can be applied to general problems in statistics, machine learning and pattern recognition. While MDL was originally based on data compression ideas, this introduction can be read without any knowledge thereof. It takes into account all major developments since 2007, the last time an extensive overview was written. These include new methods for model selection and averaging and hypothesis testing, as well as the first completely general definition of {\em MDL estimators}. Incorporating these developments, MDL can be seen as a powerful extension of both penalized likelihood and Bayesian approaches, in which penalization functions and prior distributions are replaced by more general luckiness functions, average-case methodology is replaced by a more robust worst-case approach, and in which methods classically viewed as highly distinct, such as AIC vs BIC and cross-validation vs Bayes can, to a large extent, be viewed from a unified perspective.

arXiv commentto appear in International Journal of Mathematics for Industry

References in corpus (2)

• From $\epsilon$-entropy to KL-entropy: Analysis of minimum information complexity density estimation
• Finding Good Itemsets by Packing Data

Cited by in corpus (16)

• The no-free-lunch theorems of supervised learning
• Marginal likelihood computation for model selection and hypothesis testing: an extensive review
• Exhaustive Symbolic Regression
• Community Detection in Bipartite Networks with Stochastic Blockmodels
• On the functional form of the radial acceleration relation
• Robust subgroup discovery
• Discovering outstanding subgroup lists for numeric targets using MDL
• An experimental evaluation of choices of SSA forecasting parameters
• Multiclass classification utilising an estimated algorithmic probability prior
• Far from Asymptopia
• Compression-based inference of network motif sets
• A Geometric Modeling of Occam's Razor in Deep Learning
• Constraining dark matter halo profiles with symbolic regression
• Description length of canonical and microcanonical models
• Selecting fitted models under epistemic uncertainty using a stochastic process on quantile functions
• On Confidence Sequences for Bounded Random Processes via Universal Gambling Strategies