9 papers · 1 filter
CKV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference
Chuheng Du, Junyi Chen, Hanlin Tang +7
Long-context inference is central to modern large language model (LLM) applications such as retrieval-augmented generation and multi-document reasoning. To mitigate the growing inf…
SmartThinker: Progressive Chain-of-Thought Length Calibration for Efficient Large Language Model Reasoning
Chenzhi Hu, Qinzhe Hu, Yuhang Xu +6
Large reasoning models (LRMs) like OpenAI o1 and DeepSeek-R1 achieve high accuracy on complex tasks by adopting long chain-of-thought (CoT) reasoning paths. However, the inherent v…
Learning to Seek Help: Dynamic Collaboration Between Small and Large Language Models
Hang Zeng, Xiangyu Liu, Yong Hu +5
Large language models (LLMs) offer strong capabilities but raise cost and privacy concerns, whereas small language models (SLMs) facilitate efficient and private local inference ye…
From Myopic Selection to Long-Horizon Awareness: Sequential LLM Routing for Multi-Turn Dialogue
Jiarui Zhang, Xiangyu Liu, Yong Hu +5
Multi-turn dialogue is the predominant form of interaction with large language models (LLMs). While LLM routing is effective in single-turn settings, existing methods fail to maxim…
CORE: Reducing UI Exposure in Mobile Agents via Collaboration Between Cloud and Local LLMs
Gucongcong Fan, Chaoyue Niu, Chengfei Lyu +2
Mobile agents rely on Large Language Models (LLMs) to plan and execute tasks on smartphone user interfaces (UIs). While cloud-based LLMs achieve high task accuracy, they require up…
RAGRouter: Learning to Route Queries to Multiple Retrieval-Augmented Language Models
Jiarui Zhang, Xiangyu Liu, Yong Hu +3
Retrieval-Augmented Generation (RAG) significantly improves the performance of Large Language Models (LLMs) on knowledge-intensive tasks. However, varying response quality across L…