79 citations · 79 across the 1 of their papers we have counts for
5 papers
ZeroQ: A Novel Zero Shot Quantization Framework
Yaohui Cai, Zhewei Yao, Zhen Dong +3
Quantization is a promising approach for reducing the inference time and memory footprint of neural networks. However, most existing quantization methods require access to the orig…
Checkmate: Breaking the Memory Wall with Optimal Tensor Rematerialization
Paras Jain, Ajay Jain, Aniruddha Nrusimha +5
We formalize the problem of trading-off DNN training time and memory requirements as the tensor rematerialization optimization problem, a generalization of prior checkpointing stra…
HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision
Zhen Dong, Zhewei Yao, Amir Gholami +2
Model size and inference speed/power have become a major challenge in the deployment of Neural Networks for many applications. A promising approach to address these problems is qua…
ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs
Amir Gholami, Kurt Keutzer, George Biros
Residual neural networks can be viewed as the forward Euler discretization of an Ordinary Differential Equation (ODE) with a unit time step. This has recently motivated researchers…
On the Computational Inefficiency of Large Batch Sizes for Stochastic Gradient Descent
Noah Golmant, Nikita Vemuri, Zhewei Yao +5
Increasing the mini-batch size for stochastic gradient descent offers significant opportunities to reduce wall-clock training time, but there are a variety of theoretical and syste…