118 citations · 1.4k across the 52 of their papers we have counts for
8 papers · 1 filter
PyHessian: Neural Networks Through the Lens of the Hessian
Zhewei Yao, Amir Gholami, Kurt Keutzer +1
We present PYHESSIAN, a new scalable framework that enables fast computation of Hessian (i.e., second-order derivative) information for deep neural networks. PYHESSIAN enables fast…
HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks
Zhen Dong, Zhewei Yao, Yaohui Cai +4
Quantization is an effective method for reducing memory footprint and inference time of Neural Networks, e.g., for efficient inference in the cloud, especially at the edge. However…
Checkmate: Breaking the Memory Wall with Optimal Tensor Rematerialization
Paras Jain, Ajay Jain, Aniruddha Nrusimha +5
We formalize the problem of trading-off DNN training time and memory requirements as the tensor rematerialization optimization problem, a generalization of prior checkpointing stra…
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT
Sheng Shen, Zhen Dong, Jiayu Ye +5
Transformer based architectures have become de-facto models used for a range of Natural Language Processing tasks. In particular, the BERT based models achieved significant accurac…
ANODEV2: A Coupled Neural ODE Evolution Framework
Tianjun Zhang, Zhewei Yao, Amir Gholami +4
It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE). This observation motivated the introducti…
HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision
Zhen Dong, Zhewei Yao, Amir Gholami +2
Model size and inference speed/power have become a major challenge in the deployment of Neural Networks for many applications. A promising approach to address these problems is qua…