2 papers
cs.CL2026
Less is More: Lightweight Prompt Compression for Question Answering Applications on Edge Devices
Zihuai Xu, Ruofei Hou, Yang Xu +3
In agent-driven question answering (QA) applications, retrieval-augmented generation (RAG) is commonly introduced to enhance the response accuracy of large language models (LLMs) b…
cs.DC2025
Collaborative Speculative Inference for Efficient LLM Inference Serving
Luyao Gao, Jianchun Liu, Hongli Xu +3
Speculative inference is a promising paradigm employing small speculative models (SSMs) as drafters to generate draft tokens, which are subsequently verified in parallel by the tar…