Training and inference of large language models using 8-bit floating point

Sergio P. Perez, Yan Zhang, James Briggs +6

FP8 formats are gaining popularity to boost the computational efficiency for training and inference of large deep learning models. Their main challenge is that a careful choice of…