14 citations · 48 across the 7 of their papers we have counts for
7 papers
An Analytical Characterization of Sloppiness in Neural Networks: Insights from Linear Models
Jialin Mao, Itay Griniasty, Yan Sun +3
Recent experiments have shown that training trajectories of multiple deep neural networks with different architectures, optimization algorithms, hyper-parameter settings, and regul…
The Training Process of Many Deep Networks Explores the Same Low-Dimensional Manifold
Jialin Mao, Itay Griniasty, Han Kheng Teoh +5
We develop information-geometric techniques to analyze the trajectories of the predictions of deep networks during training. By examining the underlying high-dimensional probabilis…
A picture of the space of typical learnable tasks
Rahul Ramesh, Jialin Mao, Itay Griniasty +5
We develop information geometric techniques to understand the representations learned by deep networks when they are trained on different tasks using supervised, meta-, semi-superv…
Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacy
Zhiqi Bu, Jialin Mao, Shiyun Xu
Large convolutional neural networks (CNN) can be difficult to train in the differentially private (DP) regime, since the optimization algorithms require a computationally expensive…
Does the Data Induce Capacity Control in Deep Learning?
Rubing Yang, Jialin Mao, Pratik Chaudhari
We show that the input correlation matrix of typical classification datasets has an eigenspectrum where, after a sharp initial drop, a large number of small eigenvalues are distrib…
Saliency-based Sequential Image Attention with Multiset Prediction
Sean Welleck, Jialin Mao, Kyunghyun Cho +1
Humans process visual scenes selectively and sequentially using attention. Central to models of human visual attention is the saliency map. We propose a hierarchical visual archite…