FP4 All the Way: Fully Quantized Training of LLMs

Brian Chmiel, Maxim Fishman, Ron Banner +1

We demonstrate, for the first time, fully quantized training (FQT) of large language models (LLMs) using predominantly 4-bit floating-point (FP4) precision for weights, activations…