Faster Inference of LLMs using FP8 on the Intel Gaudi

Joonhyung Lee, Shmulik Markovich-Golan, Daniel Ohayon +9

Low-precision data types are essential in modern neural networks during both training and inference as they enhance throughput and computational capacity by better exploiting avail…