259 citations · 1.2k across the 48 of their papers we have counts for
7 papers · 1 filter
HAWQV3: Dyadic Neural Network Quantization
Zhewei Yao, Zhen Dong, Zhangcheng Zheng +8
Current low-precision quantization algorithms often have the hidden cost of conversion back and forth from floating point to quantized integer values. This hidden cost limits the l…
A Statistical Framework for Low-bitwidth Training of Deep Neural Networks
Jianfei Chen, Yu Gai, Zhewei Yao +2
Fully quantized training (FQT), which uses low-bitwidth hardware by quantizing the activations, weights, and gradients of a neural network model, is a promising approach to acceler…
MAF: Multimodal Alignment Framework for Weakly-Supervised Phrase Grounding
Qinxin Wang, Hao Tan, Sheng Shen +2
Phrase localization is a task that studies the mapping from textual phrases to regions of an image. Given difficulties in annotating phrase-to-object datasets at scale, we develop…
Improving Semi-supervised Federated Learning by Reducing the Gradient Diversity of Models
Zhengming Zhang, Yaoqing Yang, Zhewei Yao +3
Federated learning (FL) is a promising way to use the computing power of mobile devices while maintaining the privacy of users. Current work in FL, however, makes the unrealistic a…
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
Zhewei Yao, Amir Gholami, Sheng Shen +3
We introduce ADAHESSIAN, a second order stochastic optimization algorithm which dynamically incorporates the curvature of the loss function via ADAptive estimates of the HESSIAN. S…
PowerNorm: Rethinking Batch Normalization in Transformers
Sheng Shen, Zhewei Yao, Amir Gholami +2
The standard normalization method for neural network (NN) models used in Natural Language Processing (NLP) is layer normalization (LN). This is different than batch normalization (…
ZeroQ: A Novel Zero Shot Quantization Framework
Yaohui Cai, Zhewei Yao, Zhen Dong +3
Quantization is a promising approach for reducing the inference time and memory footprint of neural networks. However, most existing quantization methods require access to the orig…