HiFloat4 Format for Language Model Inference

Yuanyong Luo, Jing Huang, Yu Cheng +19

This paper introduces HiFloat4 (HiF4), a block floating-point data format tailored for deep learning. Each HiF4 unit packs 64 4-bit elements with 32 bits of shared scaling metadata…