55 citations · 102 across the 14 of their papers we have counts for
14 papers
Deterministic Bounds and Random Estimates of Metric Tensors on Neuromanifolds
Ke Sun
The high-dimensional parameter space of deep neural networks -- the neuromanifold -- is endowed with a unique metric tensor defined by the Fisher information. Reliable and scalable…
Trade-Offs of Diagonal Fisher Information Matrix Estimators
Alexander Soen, Ke Sun
The Fisher information matrix can be used to characterize the local geometry of the parameter space of neural networks. It elucidates insightful theories and useful tools to unders…
Transformed Distribution Matching for Missing Value Imputation
He Zhao, Ke Sun, Amir Dezfouli +1
We study the problem of imputing missing values in a dataset, which has important applications in many domains. The key to missing value imputation is to capture the data distribut…
On the Variance of the Fisher Information for Deep Learning
Alexander Soen, Ke Sun
In the realm of deep learning, the Fisher information matrix (FIM) gives novel insights and useful tools to characterize the loss landscape, perform second-order optimization, and…
A Geometric Modeling of Occam's Razor in Deep Learning
Ke Sun, Frank Nielsen
Why do deep neural networks (DNNs) benefit from very high dimensional parameter spaces? Their huge parameter complexities vs stunning performance in practice is all the more intrig…
Fisher-Bures Adversary Graph Convolutional Networks
Ke Sun, Piotr Koniusz, Zhen Wang
In a graph convolutional network, we assume that the graph is generated wrt some observation noise. During learning, we make small random perturbations of the graph and tr…