5 papers
Fitting Is Not Enough: Smoothness in Extremely Quantized LLMs
Yuzhuang Xu, Xu Han, Yuxuan Li +2
Large language models (LLMs) achieve strong performance but incur high deployment costs, motivating extremely low-bit but lossy quantization. Existing quantization algorithms mainl…
HUOZIIME: An On-Device LLM-enhanced Input Method for Deep Personalization
Baocai Shan, Yuzhuang Xu, Wanxiang Che
Mobile input method editors (IMEs) are the primary interface for text input, yet they remain constrained to manual typing and struggle to produce personalized text. While lightweig…
ArcLight: A Lightweight LLM Inference Architecture for Many-Core CPUs
Yuzhuang Xu, Xu Han, Yuxuan Li +1
Although existing frameworks for large language model (LLM) inference on CPUs are mature, they fail to fully exploit the computation potential of many-core CPU platforms. Many-core…
AI Security Beyond Core Domains: Resume Screening as a Case Study of Adversarial Vulnerabilities in Specialized LLM Applications
Honglin Mu, Jinghao Liu, Kaiyang Wan +4
Large Language Models (LLMs) excel at text comprehension and generation, making them ideal for automated tasks like code review and content moderation. However, our research identi…
Concise and Precise Context Compression for Tool-Using Language Models
Yang Xu, Yunlong Feng, Honglin Mu +9
Through reading the documentation in the context, tool-using language models can dynamically extend their capability using external tools. The cost is that we have to input lengthy…