34 citations · 41 across the 5 of their papers we have counts for
5 papers
EXAQ: Exponent Aware Quantization For LLMs Acceleration
Moran Shkolnik, Maxim Fishman, Brian Chmiel +3
Quantization has established itself as the primary approach for decreasing the computational and storage expenses associated with Large Language Models (LLMs) inference. The majori…
DropCompute: simple and more robust distributed synchronous training via compute variance reduction
Niv Giladi, Shahar Gottlieb, Moran Shkolnik +5
Background: Distributed training is essential for large scale training of deep neural networks (DNNs). The dominant methods for large scale DNN training are synchronous (e.g. All-R…
Neural gradients are near-lognormal: improved quantized and sparse training
Brian Chmiel, Liad Ben-Uri, Moran Shkolnik +3
While training can mostly be accelerated by reducing the time needed to propagate neural gradients back throughout the model, most previous works focus on the quantization/pruning…
Robust Quantization: One Model to Rule Them All
Moran Shkolnik, Brian Chmiel, Ron Banner +4
Neural network quantization methods often involve simulating the quantization process during training, making the trained model highly dependent on the target bit-width and precise…
Thanks for Nothing: Predicting Zero-Valued Activations with Lightweight Convolutional Neural Networks
Gil Shomron, Ron Banner, Moran Shkolnik +1
Convolutional neural networks (CNNs) introduce state-of-the-art results for various tasks with the price of high computational demands. Inspired by the observation that spatial cor…