259 citations · 1.2k across the 48 of their papers we have counts for
10 papers · 1 filter
PyHessian: Neural Networks Through the Lens of the Hessian
Zhewei Yao, Amir Gholami, Kurt Keutzer +1
We present PYHESSIAN, a new scalable framework that enables fast computation of Hessian (i.e., second-order derivative) information for deep neural networks. PYHESSIAN enables fast…
HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks
Zhen Dong, Zhewei Yao, Yaohui Cai +4
Quantization is an effective method for reducing memory footprint and inference time of Neural Networks, e.g., for efficient inference in the cloud, especially at the edge. However…
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT
Sheng Shen, Zhen Dong, Jiayu Ye +5
Transformer based architectures have become de-facto models used for a range of Natural Language Processing tasks. In particular, the BERT based models achieved significant accurac…
ANODEV2: A Coupled Neural ODE Evolution Framework
Tianjun Zhang, Zhewei Yao, Amir Gholami +4
It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE). This observation motivated the introducti…
Residual Networks as Nonlinear Systems: Stability Analysis using Linearization
Kai Rothauge, Zhewei Yao, Zixi Hu +1
We regard pre-trained residual networks (ResNets) as nonlinear systems and use linearization, a common method used in the qualitative analysis of nonlinear systems, to understand t…
HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision
Zhen Dong, Zhewei Yao, Amir Gholami +2
Model size and inference speed/power have become a major challenge in the deployment of Neural Networks for many applications. A promising approach to address these problems is qua…
JumpReLU: A Retrofit Defense Strategy for Adversarial Attacks
N. Benjamin Erichson, Zhewei Yao, Michael W. Mahoney
It has been demonstrated that very simple attacks can fool highly-sophisticated neural network architectures. In particular, so-called adversarial examples, constructed from pertur…
Inefficiency of K-FAC for Large Batch Size Training
Linjian Ma, Gabe Montague, Jiayu Ye +4
In stochastic optimization, using large batch sizes during training can leverage parallel resources to produce faster wall-clock training times per training epoch. However, for bot…
Shallow Neural Networks for Fluid Flow Reconstruction with Limited Sensors
N. Benjamin Erichson, Lionel Mathelin, Zhewei Yao +3
In many applications, it is important to reconstruct a fluid flow field, or some other high-dimensional state, from limited measurements and limited data. In this work, we propose…
Estimating the Rate Constant from Biosensor Data via an Adaptive Variational Bayesian Approach
Y. Zhang, Z. Yao, P. Forssen +1
The means to obtain the rate constants of a chemical reaction is a fundamental open problem in both science and the industry. Traditional techniques for finding rate constants requ…