259 citations · 2.9k across the 123 of their papers we have counts for
21 papers · 1 filter
PyHessian: Neural Networks Through the Lens of the Hessian
Zhewei Yao, Amir Gholami, Kurt Keutzer +1
We present PYHESSIAN, a new scalable framework that enables fast computation of Hessian (i.e., second-order derivative) information for deep neural networks. PYHESSIAN enables fast…
Exact expressions for double descent and implicit regularization via surrogate random design
Michał Dereziński, Feynman Liang, Michael W. Mahoney
Double descent refers to the phase transition that is exhibited by the generalization error of unregularized learning models when varying the ratio between the number of parameters…
Running Alchemist on Cray XC and CS Series Supercomputers: Dask and PySpark Interfaces, Deployment Options, and Data Transfer Times
Kai Rothauge, Haripriya Ayyalasomayajula, Kristyn J. Maschhoff +2
Alchemist is a system that allows Apache Spark to achieve better performance by interfacing with HPC libraries for large-scale distributed computations. In this paper, we highlight…
Limit theorems for out-of-sample extensions of the adjacency and Laplacian spectral embeddings
Keith Levin, Fred Roosta, Minh Tang +2
Graph embeddings, a class of dimensionality reduction techniques designed for relational data, have proven useful in exploring and modeling network structure. Most dimensionality r…
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT
Sheng Shen, Zhen Dong, Jiayu Ye +5
Transformer based architectures have become de-facto models used for a range of Natural Language Processing tasks. In particular, the BERT based models achieved significant accurac…
Bootstrapping the Operator Norm in High Dimensions: Error Estimation for Covariance Matrices and Sketching
Miles E. Lopes, N. Benjamin Erichson, Michael W. Mahoney
Although the operator (spectral) norm is one of the most widely used metrics for covariance estimation, comparatively little is known about the fluctuations of error in this norm.…