185 citations · 463 across the 31 of their papers we have counts for
31 papers
ShadowNPU: System and Algorithm Co-design for NPU-Centric On-Device LLM Inference
Wangsong Yin, Daliang Xu, Mengwei Xu +2
On-device running Large Language Models (LLMs) is nowadays a critical enabler towards preserving user privacy. We observe that the attention operator falls back from the special-pu…
Elastic On-Device LLM Service
Wangsong Yin, Rongjie Yi, Daliang Xu +3
On-device Large Language Models (LLMs) are transforming mobile AI, catalyzing applications like UI automation without privacy concerns. Nowadays the common practice is to deploy a…
Fast On-device LLM Inference with NPUs
Daliang Xu, Hao Zhang, Liming Yang +4
On-device inference for Large Language Models (LLMs), driven by increasing privacy concerns and advancements of mobile-sized models, has gained significant interest. However, even…
LLM as a System Service on Mobile Devices
Wangsong Yin, Mengwei Xu, Yuanchun Li +1
Being more powerful and intrusive into user-device interactions, LLMs are eager for on-device execution to better preserve user privacy. In this work, we propose a new paradigm of…
LLMCad: Fast and Scalable On-device Large Language Model Inference
Daliang Xu, Wangsong Yin, Xin Jin +4
Generative tasks, such as text generation and question answering, hold a crucial position in the realm of mobile applications. Due to their sensitivity to privacy concerns, there i…
Mobile Foundation Model as Firmware
Jinliang Yuan, Chen Yang, Dongqi Cai +9
In today's landscape, smartphones have evolved into hubs for hosting a multitude of deep learning models aimed at local execution. A key realization driving this work is the notabl…
EdgeMoE: Empowering Sparse Large Language Models on Mobile Devices
Rongjie Yi, Liwei Guo, Shiyun Wei +3
Large language models (LLMs) such as GPTs and Mixtral-8x7B have revolutionized machine intelligence due to their exceptional abilities in generic ML tasks. Transiting LLMs from dat…
FwdLLM: Efficient FedLLM using Forward Gradient
Mengwei Xu, Dongqi Cai, Yaozong Wu +2
Large Language Models (LLMs) are transforming the landscape of mobile intelligence. Federated Learning (FL), a method to preserve user data privacy, is often employed in fine-tunin…
More is Different: Prototyping and Analyzing a New Form of Edge Server with Massive Mobile SoCs
Li Zhang, Zhe Fu, Boqing Shi +7
Huge energy consumption poses a significant challenge for edge clouds. In response to this, we introduce a new type of edge server, namely SoC Cluster, that orchestrates multiple l…
Federated Few-Shot Learning for Mobile NLP
Dongqi Cai, Shangguang Wang, Yaozong Wu +2
Natural language processing (NLP) sees rich mobile applications. To support various language understanding tasks, a foundation NLP model is often fine-tuned in a federated, privacy…
Towards Practical Few-shot Federated NLP
Dongqi Cai, Yaozong Wu, Haitao Yuan +3
Transformer-based pre-trained models have emerged as the predominant solution for natural language processing (NLP). Fine-tuning such pre-trained models for downstream tasks often…
Device-centric Federated Analytics At Ease
Li Zhang, Junji Qiu, Shangguang Wang +1
Nowadays, high-volume and privacy-sensitive data are generated by mobile devices, which are better to be preserved on devices and queried on demand. However, data analysts still la…
Mandheling: Mixed-Precision On-Device DNN Training with DSP Offloading
Daliang Xu, Mengwei Xu, Qipeng Wang +6
This paper proposes Mandheling, the first system that enables highly resource-efficient on-device training by orchestrating the mixed-precision training with on-chip Digital Signal…
Accelerating Vertical Federated Learning
Dongqi Cai, Tao Fan, Yan Kang +4
Privacy, security and data governance constraints rule out a brute force process in the integration of cross-silo data, which inherits the development of the Internet of Things. Fe…
Boosting DNN Cold Inference on Edge Devices
Rongjie Yi, Ting Cao, Ao Zhou +3
DNNs are ubiquitous on edge devices nowadays. With its increasing importance and use cases, it's not likely to pack all DNNs into device memory and expect that each inference has b…
FedAdapter: Efficient Federated Learning for Modern NLP
Dongqi Cai, Yaozong Wu, Shangguang Wang +2
Transformer-based pre-trained models have revolutionized NLP for superior performance and generality. Fine-tuning pre-trained models for downstream tasks often requires private dat…
Benchmarking of DL Libraries and Models on Mobile Devices
Qiyang Zhang, Xiang Li, Xiangying Che +6
Deploying deep learning (DL) on mobile devices has been a notable trend in recent years. To support fast inference of on-device DL, DL libraries play a critical role as algorithms…
Tiansuan Constellation: An Open Research Platform
Shangguang Wang, Qing Li, Mengwei Xu +3
Satellite network is the first step of interstellar voyages. It can provide global Internet connectivity everywhere on earth, where most areas cannot access the Internet by the ter…
Boosting Mobile CNN Inference through Semantic Memory
Yun Li, Chen Zhang, Shihao Han +4
Human brains are known to be capable of speeding up visual recognition of repeatedly presented objects through faster memory encoding and accessing procedures on activated neurons.…
Observable and Attention-Directing BDI Agents for Human-Autonomy Teaming
Blair Archibald, Muffy Calder, Michele Sevegnani +1
Human-autonomy teaming (HAT) scenarios feature humans and autonomous agents collaborating to meet a shared goal. For effective collaboration, the agents must be transparent and abl…
From Cloud to Edge: A First Look at Public Edge Platforms
Mengwei Xu, Zhe Fu, Xiao Ma +7
Public edge platforms have drawn increasing attention from both academia and industry. In this study, we perform a first-of-its-kind measurement study on a leading public edge plat…
Modelling and Verifying BDI Agents with Bigraphs
Blair Archibald, Muffy Calder, Michele Sevegnani +1
The Belief-Desire-Intention (BDI) architecture is a popular framework for rational agents; most verification approaches are based on reasoning about implementations of BDI programm…
Hierarchical Federated Learning through LAN-WAN Orchestration
Jinliang Yuan, Mengwei Xu, Xiao Ma +3
Federated learning (FL) was designed to enable mobile phones to collaboratively learn a global model without uploading their private data to a cloud server. However, exiting FL pro…
Characterizing Impacts of Heterogeneity in Federated Learning upon Large-Scale Smartphone Data
Chengxu Yang, Qipeng Wang, Mengwei Xu +4
Federated learning (FL) is an emerging, privacy-preserving machine learning paradigm, drawing tremendous attention in both academia and industry. A unique characteristic of FL is h…