74 citations · 108 across the 6 of their papers we have counts for
4 papers · 1 filter
Tensor-Parallelism with Partially Synchronized Activations
Itay Lamprecht, Asaf Karnieli, Yair Hanani +2
Training and inference of Large Language Models (LLMs) with tensor-parallelism requires substantial communication to synchronize activations. Our findings suggest that with a few m…
Dual Precision Quantization for Efficient and Accurate Deep Neural Networks Inference
Tomer Gafni, Asaf Karnieli, Yair Hanani
Deep neural networks have achieved state-of-the-art results in a wide range of applications, from natural language processing and computer vision to speech recognition. However, as…
Automatic mixed precision for optimizing gained time with constrained loss mean-squared-error based on model partition to sequential sub-graphs
Shmulik Markovich-Golan, Daniel Ohayon, Itay Niv +1
Quantization is essential for Neural Network (NN) compression, reducing model size and computational demands by using lower bit-width data types, though aggressive reduction often…
Faster Inference of LLMs using FP8 on the Intel Gaudi
Joonhyung Lee, Shmulik Markovich-Golan, Daniel Ohayon +9
Low-precision data types are essential in modern neural networks during both training and inference as they enhance throughput and computational capacity by better exploiting avail…