5 papers
ClawMobile: Rethinking Smartphone-Native Agentic Systems
Hongchao Du, Shangyu Wu, Qiao Li +4
Smartphones represent a uniquely challenging environment for agentic systems. Unlike cloud or desktop settings, mobile devices combine constrained execution contexts, fragmented co…
LAPS: A Length-Aware-Prefill LLM Serving System
Jianshu She, Zonghang Li, Hongchao Du +7
LAPS identifies and disaggregates requests with different prompt lengths in LLM serving to reduce TTFT latency. While recent systems have decoupled the prefill and decode stages to…
EvoP: Robust LLM Inference via Evolutionary Pruning
Shangyu Wu, Hongchao Du, Ying Xiong +4
Large Language Models (LLMs) have achieved remarkable success in natural language processing tasks, but their massive size and computational demands hinder their deployment in reso…
FlexInfer: Breaking Memory Constraint via Flexible and Efficient Offloading for On-Device LLM Inference
Hongchao Du, Shangyu Wu, Arina Kharlamova +2
Large Language Models (LLMs) face challenges for on-device inference due to high memory demands. Traditional methods to reduce memory usage often compromise performance and lack ad…
When Compression Meets Model Compression: Memory-Efficient Double Compression for Large Language Models
Weilan Wang, Yu Mao, Dongdong Tang +3
Large language models (LLMs) exhibit excellent performance in various tasks. However, the memory requirements of LLMs present a great challenge when deploying on memory-limited dev…