Is Finer Better? The Limits of Microscaling Formats in Large Language Models

Andrea Fasoli, Monodeep Kar, Chi-Chun Liu +4

Microscaling data formats leverage per-block tensor quantization to enable aggressive model compression with limited loss in accuracy. Unlocking their potential for efficient train…