118 citations · 1.4k across the 52 of their papers we have counts for
5 papers · 1 filter
HAWQV3: Dyadic Neural Network Quantization
Zhewei Yao, Zhen Dong, Zhangcheng Zheng +8
Current low-precision quantization algorithms often have the hidden cost of conversion back and forth from floating point to quantized integer values. This hidden cost limits the l…
Boundary thickness and robustness in learning models
Yaoqing Yang, Rajiv Khanna, Yaodong Yu +5
Robustness of machine learning models to various adversarial and non-adversarial corruptions continues to be of interest. In this paper, we introduce the notion of the boundary thi…
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
Zhewei Yao, Amir Gholami, Sheng Shen +3
We introduce ADAHESSIAN, a second order stochastic optimization algorithm which dynamically incorporates the curvature of the loss function via ADAptive estimates of the HESSIAN. S…
PowerNorm: Rethinking Batch Normalization in Transformers
Sheng Shen, Zhewei Yao, Amir Gholami +2
The standard normalization method for neural network (NN) models used in Natural Language Processing (NLP) is layer normalization (LN). This is different than batch normalization (…
ZeroQ: A Novel Zero Shot Quantization Framework
Yaohui Cai, Zhewei Yao, Zhen Dong +3
Quantization is a promising approach for reducing the inference time and memory footprint of neural networks. However, most existing quantization methods require access to the orig…