NF4 Isn't Information Theoretically Optimal (and that's Good)

Davis Yoshida

This note shares some simple calculations and experiments related to absmax-based blockwise quantization, as used in Dettmers et al., 2023. Their proposed NF4 data type is said to…