SmartQuant: CXL-based AI Model Store in Support of Runtime Configurable Weight Quantization

Rui Xie, Asad Ul Haq, Linsen Ma +5

Recent studies have revealed that, during the inference on generative AI models such as transformer, the importance of different weights exhibits substantial context-dependent vari…