14 citations · 48 across the 7 of their papers we have counts for
6 papers · 1 filter
An Analytical Characterization of Sloppiness in Neural Networks: Insights from Linear Models
Jialin Mao, Itay Griniasty, Yan Sun +3
Recent experiments have shown that training trajectories of multiple deep neural networks with different architectures, optimization algorithms, hyper-parameter settings, and regul…
The Training Process of Many Deep Networks Explores the Same Low-Dimensional Manifold
Jialin Mao, Itay Griniasty, Han Kheng Teoh +5
We develop information-geometric techniques to analyze the trajectories of the predictions of deep networks during training. By examining the underlying high-dimensional probabilis…
A picture of the space of typical learnable tasks
Rahul Ramesh, Jialin Mao, Itay Griniasty +5
We develop information geometric techniques to understand the representations learned by deep networks when they are trained on different tasks using supervised, meta-, semi-superv…
Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacy
Zhiqi Bu, Jialin Mao, Shiyun Xu
Large convolutional neural networks (CNN) can be difficult to train in the differentially private (DP) regime, since the optimization algorithms require a computationally expensive…
Does the Data Induce Capacity Control in Deep Learning?
Rubing Yang, Jialin Mao, Pratik Chaudhari
We show that the input correlation matrix of typical classification datasets has an eigenspectrum where, after a sharp initial drop, a large number of small eigenvalues are distrib…
Loss Functions for Multiset Prediction
Sean Welleck, Zixin Yao, Yu Gai +3
We study the problem of multiset prediction. The goal of multiset prediction is to train a predictor that maps an input to a multiset consisting of multiple items. Unlike existing…