7 papers
Minimum enclosing Bregman balls made easy
Frank Nielsen
In this work, we revisit the problem of computing minimum enclosing Bregman balls (Bregman MEBs) of finite sets of parameters. First, we show that Bregman MEBs are equivalent to ME…
Characterizing Learning Dynamics under Relative Reparameterization of Singular Models
Pascal Mattia Esser, Frank Nielsen
A common way to analyze learning of statistical models is to consider operations in the models parameter space, however this becomes challenging when there is no one-to-one mapping…
A Geometric Modeling of Occam's Razor in Deep Learning
Ke Sun, Frank Nielsen
Why do deep neural networks (DNNs) benefit from very high dimensional parameter spaces? Their huge parameter complexities vs stunning performance in practice is all the more intrig…
Beyond scalar quasi-arithmetic means: Quasi-arithmetic averages and quasi-arithmetic mixtures in information geometry
Frank Nielsen
We generalize quasi-arithmetic means beyond scalars by considering the gradient map of a Legendre type real-valued function. The gradient map of a Legendre type function is proven…
The duo Bregman and Fenchel-Young divergences
Frank Nielsen
By calculating the Kullback-Leibler divergence between two probability measures belonging to different exponential families, we end up with a formula that generalizes the ordinary…
Information measures and geometry of the hyperbolic exponential families of Poincaré and hyperboloid distributions
Frank Nielsen, Kazuki Okamura
We study various information-theoretic measures and the information geometry of the Poincaré distributions and the related hyperboloid distributions, and prove that their statisti…