32 citations · 50 across the 5 of their papers we have counts for
9 papers
EdgeCraft: Automated Model Crafting for Edge IoT
Genglin Wang, Kaiwei Liu, Liekang Zeng +4
Machine learning (ML) increasingly powers Internet of Things (IoT) applications at the edge. Yet producing a deployable edge ML artifact for a specific scenario requires navigating…
CacheRepair: Learning to Repair Cross-Chunk Context in RAG for KV Cache Fusion
Genglin Wang, Wangsong Yin, Yeerzhati Abudunuer +3
Multi-document retrieval-augmented generation (RAG) requires a language model to process multiple retrieved text chunks before answering a question. Precomputing each chunk's KV ca…
OmniTide: Co-Designing Algorithms and Systems for Efficient On-Device Omni-LLM Streaming
Zongshang Shen, Wangsong Yin, Daliang Xu +2
On-device streaming omni-modal inference safeguards user privacy and eliminates prohibitive per-token API costs, but faces a critical bottleneck: the continuous influx of multimoda…
Empowering Hybrid Attention Models on NPUs
Yinyuan Zhang, Daliang Xu, Xiaolong Huang +4
Hybrid attention models have emerged as a crucial architecture for Large Language Models (LLMs) (e.g., the Qwen3.5 and Kimi series). Their memory and computational efficiency make…
ShadowNPU: System and Algorithm Co-design for NPU-Centric On-Device LLM Inference
Wangsong Yin, Daliang Xu, Mengwei Xu +2
On-device running Large Language Models (LLMs) is nowadays a critical enabler towards preserving user privacy. We observe that the attention operator falls back from the special-pu…
Elastic On-Device LLM Service
Wangsong Yin, Rongjie Yi, Daliang Xu +3
On-device Large Language Models (LLMs) are transforming mobile AI, catalyzing applications like UI automation without privacy concerns. Nowadays the common practice is to deploy a…
LLM as a System Service on Mobile Devices
Wangsong Yin, Mengwei Xu, Yuanchun Li +1
Being more powerful and intrusive into user-device interactions, LLMs are eager for on-device execution to better preserve user privacy. In this work, we propose a new paradigm of…
A Survey of Resource-efficient LLM and Multimodal Foundation Models
Mengwei Xu, Wangsong Yin, Dongqi Cai +15
Large foundation models, including large language models (LLMs), vision transformers (ViTs), diffusion, and LLM-based multimodal models, are revolutionizing the entire machine lear…
LLMCad: Fast and Scalable On-device Large Language Model Inference
Daliang Xu, Wangsong Yin, Xin Jin +4
Generative tasks, such as text generation and question answering, hold a crucial position in the realm of mobile applications. Due to their sensitivity to privacy concerns, there i…