Training an MoE LLM in MXFP4: How We Closed the Gap to BF16

Han Wang, Zhitao Wang, Natalia Frumkin +4

Training large language models (LLMs) in sub-8-bit arithmetic promises substantial gains in memory bandwidth and compute throughput, yet FP4 formats introduce quantization error se…