12 papers
LASER: Load-Aware Serving with Early-Exit for Reasoning LLMs at the Edge
Zhiqing Tang, Size Li, Hanshuai Cui +5
Large reasoning models (LRMs) such as DeepSeek-R1 have achieved strong performance through extended chain-of-thought (CoT) generation. However, deploying them on edge devices raise…
RISE: Relay Inference and Online Scheduling for Efficient Edge-Device Collaborative Diffusion Model Services
Zilan Huang, Zhiqing Tang, Hanshuai Cui +4
Text-to-image diffusion models are increasingly deployed at the network edge to serve heterogeneous workloads with diverse quality and latency requirements. However, existing deplo…
ConCise: Training-Free Conclusion-Chain State Compression for Cost-Efficient Multi-Step RAG Services
Kuan Yan, Zhiqing Tang, Tian Wang +1
Multi-step retrieval-augmented generation (RAG) has been widely deployed as LLM-powered web services for complex question answering, where iterative retrieval-reasoning rounds deli…
CONCORD: Asynchronous Sparse Aggregation for Device-Cloud RAG under Document Isolation
Xuedong Hu, Zhiqing Tang, Zhi Yao +2
Retrieval-augmented generation (RAG) has emerged as a pivotal technique for improving language models by incorporating external knowledge at inference time. As device-cloud collabo…
Ling and Ring 2.6 Technical Report: Efficient and Instant Agentic Intelligence at Trillion-Parameter Scale
Ang Li, Ben Liu, Bin Han +215
Efficient and scalable agentic intelligence requires models that can deliver both low-latency responses and strong reasoning capabilities while remaining practical to train, serve,…
HeraSys: Collaborative Serving of Multiple LLM Workflows via Fine-Grained End-to-End Optimization
Size Li, Zhiqing Tang, Hongrui Liang +4
The proliferation of Large Language Models (LLMs) has shifted serving systems from processing isolated requests to orchestrating high-concurrency, multi-tenant agentic workflows. H…