SmoothQuant+: Accurate and Efficient 4-bit Post-Training WeightQuantization for LLM

Jiayi Pan, Chengcan Wang, Kaifu Zheng +3

Large language models (LLMs) have shown remarkable capabilities in various tasks. However their huge model size and the consequent demand for computational and memory resources als…