2 papers
cs.LG2024
Rotated Runtime Smooth: Training-Free Activation Smoother for accurate INT4 inference
Ke Yi, Zengke Liu, Jianwei Zhang +4
Large language models have demonstrated promising capabilities upon scaling up parameters. However, serving large language models incurs substantial computation and memory movement…
cs.AI2024
One QuantLLM for ALL: Fine-tuning Quantized LLMs Once for Efficient Deployments
Ke Yi, Yuhui Xu, Heng Chang +4
Large Language Models (LLMs) have advanced rapidly but face significant memory demands. While quantization has shown promise for LLMs, current methods typically require lengthy tra…