A scale-dependent notion of effective dimension
arXiv:2001.10872
Abstract
We introduce a notion of "effective dimension" of a statistical model based on the number of cubes of size needed to cover the model space when endowed with the Fisher Information Matrix as metric, being the number of observations. The number of observations fixes a natural scale or resolution. The effective dimension is then measured via the spectrum of the Fisher Information Matrix regularized using this natural scale.