SiLQ: Simple Large Language Model Quantization-Aware Training

Steven K. Esser, Jeffrey L. McKinstry, Deepika Bablani +2

Large language models can be quantized to reduce inference time latency, model size, and energy consumption, thereby delivering a better user experience at lower cost. A challenge…