7 papers
PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud
Chenghua Wang, Daliang Xu, Dongqi Cai +24
Physical AI policies require inference throughout their lifecycle, including model evaluation, cloud reinforcement learning rollout, edge GPU serving, and onboard deployment. Altho…
NanoSpec: Accelerating Speculative Decoding using Minimalist In-Context Vocabularies
Zhiyang Chen, Daliang Xu, Yinyuan Zhang +3
The massive vocabulary sizes of large language models, often exceeding 100k tokens, impose a computational bottleneck on the final linear projection layer during speculative decodi…
Quant.npu: Enabling Efficient Mobile NPU Inference for on-device LLMs via Fully Static Quantization
Jinghe Zhang, Daliang Xu, Chenghua Wang +5
Large language models (LLMs) are increasingly deployed on mobile devices, where Neural Processing Units (NPUs) necessitate fully static quantization for optimal inference efficienc…
Accelerating Mobile Language Model via Speculative Decoding and NPU-Coordinated Execution
Zhiyang Chen, Daliang Xu, Haiyang Shen +5
Performing Retrieval-Augmented Generation (RAG) directly on mobile devices is promising for data privacy and responsiveness but is hindered by the architectural constraints of mobi…
MobiEdit: Resource-efficient Knowledge Editing for Personalized On-device LLMs
Zhenyan Lu, Daliang Xu, Dongqi Cai +5
Large language models (LLMs) are deployed on mobile devices to power killer applications such as intelligent assistants. LLMs pre-trained on general corpora often hallucinate when…
LoRASuite: Efficient LoRA Adaptation Across Large Language Model Upgrades
Yanan Li, Fanxu Meng, Muhan Zhang +3
As Large Language Models (LLMs) are frequently updated, LoRA weights trained on earlier versions quickly become obsolete. The conventional practice of retraining LoRA weights from…