5 papers
Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantization
Vage Egiazarian, Roberto L. Castro, Denis Kuznedelev +8
The recent hardware-accelerated microscaling 4-bit floating-point formats such as MXFP4 and NVFP4, supported on NVIDIA and AMD GPUs, promise to revolutionize large language model (…
Quartet: Native FP4 Training Can Be Optimal for Large Language Models
Roberto L. Castro, Andrei Panferov, Soroush Tabesh +5
Training large language models (LLMs) models directly in low-precision offers a way to address computational costs by improving both throughput and energy efficiency. For those pur…
HALO: Hadamard-Assisted Lower-Precision Optimization for LLMs
Saleh Ashkboos, Mahdi Nikdan, Soroush Tabesh +3
Quantized training of Large Language Models (LLMs) remains an open challenge, as maintaining accuracy while performing all matrix multiplications in low precision has proven diffic…
Beyond Outliers: A Study of Optimizers Under Quantization
Georgios Vlassis, Saleh Ashkboos, Alexandra Volkova +2
As new optimizers gain traction and model quantization becomes standard for efficient deployment, a key question arises: how does the choice of optimizer affect model performance i…
Computational Bottlenecks of Training Small-scale Large Language Models
Saleh Ashkboos, Iman Mirzadeh, Keivan Alizadeh +4
While large language models (LLMs) dominate the AI landscape, Small-scale large Language Models (SLMs) are gaining attention due to cost and efficiency demands from consumers. Howe…