5 papers
MosaicKV: Serving Long-Context LLM with Dynamic Two-D KV Cache Compression
Sheng Qiang, Ruiwei Chen, Yinpeng Wu +5
Long-context LLM services now sustain prompts with hundreds of thousands to millions of tokens, making the key-value (KV) cache a first-order serving cost. Because the cache grows…
FlexServe: A Fast and Secure LLM Serving System for Mobile Devices with Flexible Resource Isolation
Yinpeng Wu, Yitong Chen, Lixiang Wang +3
Device-side Large Language Models (LLMs) have grown explosively, offering stronger privacy and higher availability than their cloud-side counterparts. During LLM inference, both th…
FlexServe: A Fast and Secure LLM Serving System for Mobile Devices with Flexible Resource Isolation
Yinpeng Wu, Yitong Chen, Lixiang Wang +3
Device-side Large Language Models (LLMs) have witnessed explosive growth, offering higher privacy and availability compared to cloud-side LLMs. During LLM inference, both model wei…
Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment
Kaijun Zhou, Qiwei Chen, Da Peng +3
Vision-Language-Action (VLA) models are promising for generalist robot control, but on-robot deployment is bottlenecked by real-time inference under tight cost and energy budgets.…
TZ-LLM: Protecting On-Device Large Language Models with Arm TrustZone
Xunjie Wang, Jiacheng Shi, Zihan Zhao +3
Large Language Models (LLMs) deployed on mobile devices offer benefits like user privacy and reduced network latency, but introduce a significant security risk: the leakage of prop…