8 papers
Max-Affine Spline Insights Into Deep Network Pruning
Haoran You, Randall Balestriero, Zhihan Lu +6
In this paper, we study the importance of pruning in Deep Networks (DNs) and the yin & yang relationship between (1) pruning highly overparametrized DNs that have been trained from…
Drawing Early-Bird Tickets: Towards More Efficient Training of Deep Networks
Haoran You, Chaojian Li, Pengfei Xu +6
(Frankle & Carbin, 2019) shows that there exist winning tickets (small but critical subnetworks) for dense, randomly initialized networks, that can be trained alone to achieve comp…
A Primal-Dual Framework for Transformers and Neural Networks
Tan M. Nguyen, Tam Nguyen, Nhat Ho +3
Self-attention is key to the remarkable success of transformers in sequence modeling tasks including many applications in natural language processing and computer vision. Like neur…
SplineCam: Exact Visualization and Characterization of Deep Network Geometry and Decision Boundaries
Ahmed Imtiaz Humayun, Randall Balestriero, Guha Balakrishnan +1
Current Deep Network (DN) visualization and interpretability methods rely heavily on data space visualizations such as scoring which dimensions of the data are responsible for thei…
The Common Intuition to Transfer Learning Can Win or Lose: Case Studies for Linear Regression
Yehuda Dar, Daniel LeJeune, Richard G. Baraniuk
We study a fundamental transfer learning process from source to target linear regression tasks, including overparameterized settings where there are more learned parameters than da…
TITAN: Bringing The Deep Image Prior to Implicit Representations
Lorenzo Luzi, Daniel LeJeune, Ali Siahkoohi +6
We study the interpolation capabilities of implicit neural representations (INRs) of images. In principle, INRs promise a number of advantages, such as continuous derivatives and a…