Arithmetic-Intensity-Aware Quantization

Taig Singh, Shreshth Rajan, Nikhil Jain

As modern neural networks become increasingly memory-bound, inference throughput is limited by DRAM bandwidth rather than compute. We present Arithmetic-Intensity-Aware Quantizatio…