9 citations · 13 across the 10 of their papers we have counts for
4 papers · 1 filter
Specter: Fast MoE Inference via Quantization-Enabled Multi-Step Speculative Offloading
Wenfeng Wang, Jiacheng Liu, Xiaofeng Hou +7
Mixture-of-experts (MoE) models often exceed a single accelerator's memory capacity. Offloading experts to host memory creates a severe PCIe I/O bottleneck because data-dependent r…
MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs
Xinfeng Xia, Xiaofeng Hou, Jiacheng Liu +5
Mixture-of-Experts (MoE) scales model capacity through sparse activation, and is becoming an important architecture for large language models (LLMs). However, existing MoE serving…
Survey of Disaggregated Memory: Cross-layer Technique Insights for Next-Generation Datacenters
Jing Wang, Chao Li, Taolei Wang +4
The growing scale of data requires efficient memory subsystems with large memory capacity and high memory performance. Disaggregated architecture has become a promising solution fo…
A Survey on Inference Optimization Techniques for Mixture of Experts Models
Jiacheng Liu, Peng Tang, Wenfeng Wang +5
The emergence of large-scale Mixture of Experts (MoE) models represents a significant advancement in artificial intelligence, offering enhanced model capacity and computational eff…