FOCUS: FP4 Optimization via Coupled-Relaxation and Dual-Granularity Scaling

Xianglong Yan, Hong Liu, Chengzhu Bao +4

Large language models (LLMs) achieve remarkable performance but are expensive to deploy due to their enormous size. FP4 quantization, with formats such as MXFP4 and NVFP4, offers a…