EXAQ: Exponent Aware Quantization For LLMs Acceleration

Moran Shkolnik, Maxim Fishman, Brian Chmiel +3

Quantization has established itself as the primary approach for decreasing the computational and storage expenses associated with Large Language Models (LLMs) inference. The majori…