What Matters for NVFP4 Training? A Comparative Study of Low-Precision Pretraining Recipes

Anjulie Agrusa, Andrei Panferov, Elizabeth Wei +7

Training large language models directly in 4-bit floating-point (FP) formats promises substantial improvements in throughput and energy efficiency. While some recipe design choices…