FullPack: Full Vector Utilization for Sub-Byte Quantized Inference on General Purpose CPUs

Hossein Katebi, Navidreza Asadi, Maziar Goudarzi

Although prior art has demonstrated negligible accuracy drop in sub-byte quantization -- where weights and/or activations are represented by less than 8 bits -- popular SIMD instru…