FBQuant: FeedBack Quantization for Large Language Models

Yijiang Liu, Hengyu Fang, Liulu He +4

Deploying Large Language Models (LLMs) on edge devices is increasingly important, as it eliminates reliance on network connections, reduces expensive API calls, and enhances user p…