1 citations · 1 across the 10 of their papers we have counts for
10 papers
TopoCompress: Topology Aware Token Compression Algorithm for Distributed Edge MoE Inference
Ning Li, Xinyu Wang, Xin Yuan +3
Mixture-of-experts (MoE) models improve capacity with moderate overhead by sparsely activating experts per token. However, deploying MoE across resource-constrained edge servers in…
HetRoute Heterogeneous and Cost-aware Collaborative Routing Framework for Distributed Edge MoE Inference
Xin Yuan, Ning Li, Wenchao Xu +2
Mixture-of-Experts (MoE) models have become a dominant architecture for large-scale AI services, yet deploying them over geo-distributed heterogeneous edge servers remains challeng…
TrimMoE A communication aware and adaptive depth framework for distributed edge inference
Ning Li, Shuting Bai, Xin Yuan +3
Serving Mixture-of-Experts (MoE) large language models across distributed edge servers is bottlenecked by the cross-server expert transmission. The existing approaches mainly focus…
OrderMoE: An expert similarity driven distributed edge MoE inference
Xin Yuan, Ning Li, Quan Chen +2
Although mixture-of-experts, MoE, models have been increasingly adopted to scale large language models with moderate computation cost, it remains challenging to deploy MoE inferenc…
CoMoE: Collaborative Optimization of Expert Aggregation and Offloading for MoE-based LLMs at Edge
Muqing Li, Ning Li, Xin Yuan +4
The proliferation of large language models (LLMs) has driven the adoption of Mixture-of-Experts (MoE) architectures as a promising solution to scale model capacity while controllin…
Efficient Edge LLMs Deployment via HessianAware Quantization and CPU GPU Collaborative
Tuo Zhang, Ning Li, Xin Yuan +4
With the breakthrough progress of large language models (LLMs) in natural language processing and multimodal tasks, efficiently deploying them on resource-constrained edge devices…
The MoE-Empowered Edge LLMs Deployment: Architecture, Challenges, and Opportunities
Ning Li, Song Guo, Tuo Zhang +5
The powerfulness of LLMs indicates that deploying various LLMs with different scales and architectures on end, edge, and cloud to satisfy different requirements and adaptive hetero…
A QoE-Aware Split Inference Accelerating Algorithm for NOMA-based Edge Intelligence
Xin Yuan, Ning Li, Quan Chen +3
Even the AI has been widely used and significantly changed our life, deploying the large AI models on resource limited edge devices directly is not appropriate. Thus, the model spl…
Mobility and Cost Aware Inference Accelerating Algorithm for Edge Intelligence
Xin Yuan, Ning Li, kang Wei +4
The edge intelligence (EI) has been widely applied recently. Spliting the model between device, edge server, and cloud can improve the performance of EI greatly. The model segmenta…
High Efficiency Inference Accelerating Algorithm for NOMA-based Mobile Edge Computing
Xin Yuan, Ning Li, Tuo Zhang +4
Splitting the inference model between device, edge server, and cloud can improve the performance of EI greatly. Additionally, the non-orthogonal multiple access (NOMA), which is th…