13 citations · 38 across the 5 of their papers we have counts for
5 papers
MNN-AECS: Energy Optimization for LLM Decoding on Mobile Devices via Adaptive Core Selection
Zhengxiang Huang, Chaoyue Niu, Zhaode Wang +8
As the demand for on-device Large Language Model (LLM) inference grows, energy efficiency has become a major concern, especially for battery-limited mobile devices. Our analysis sh…
MNN-LLM: A Generic Inference Engine for Fast Large Language Model Deployment on Mobile Devices
Zhaode Wang, Jingbang Yang, Xinyu Qian +4
Large language models (LLMs) have demonstrated exceptional performance across a variety of tasks. However, their substantial scale leads to significant computational resource consu…
Walle: An End-to-End, General-Purpose, and Large-Scale Production System for Device-Cloud Collaborative Machine Learning
Chengfei Lv, Chaoyue Niu, Renjie Gu +17
To break the bottlenecks of mainstream cloud-based machine learning (ML) paradigm, we adopt device-cloud collaborative ML and build the first end-to-end and general-purpose system,…
INT8 Winograd Acceleration for Conv1D Equipped ASR Models Deployed on Mobile Devices
Yiwu Yao, Yuchao Li, Chengyu Wang +8
The intensive computation of Automatic Speech Recognition (ASR) models obstructs them from being deployed on mobile devices. In this paper, we present a novel quantized Winograd op…
MNN: A Universal and Efficient Inference Engine
Xiaotang Jiang, Huan Wang, Yiliu Chen +9
Deploying deep learning models on mobile devices draws more and more attention recently. However, designing an efficient inference engine on devices is under the great challenges o…