3 papers
cs.LG2026
Deterministic Bounds and Random Estimates of Metric Tensors on Neuromanifolds
Ke Sun
The high-dimensional parameter space of deep neural networks -- the neuromanifold -- is endowed with a unique metric tensor defined by the Fisher information. Reliable and scalable…
cs.LG2025
A Geometric Modeling of Occam's Razor in Deep Learning
Ke Sun, Frank Nielsen
Why do deep neural networks (DNNs) benefit from very high dimensional parameter spaces? Their huge parameter complexities vs stunning performance in practice is all the more intrig…
cs.LG2024
Trade-Offs of Diagonal Fisher Information Matrix Estimators
Alexander Soen, Ke Sun
The Fisher information matrix can be used to characterize the local geometry of the parameter space of neural networks. It elucidates insightful theories and useful tools to unders…