HALO: Hardware-aware quantization with low critical-path-delay weights for LLM acceleration

Rohan Juneja, Shivam Aggarwal, Safeen Huda +2

Quantization is critical for efficiently deploying large language models (LLMs). Yet conventional methods remain hardware-agnostic, limited to bit-width constraints, and do not acc…