20 citations · 32 across the 9 of their papers we have counts for
7 papers · 1 filter
Auto-Scaling Heterogeneous Neural Processing Units for Energy and Cost-Efficient LLM Serving
Yuqi Xue, Jichuan Chang, Jian Huang
To meet the ever-increasing computing demands of large language model (LLM) services, modern cloud platforms have widely deployed neural processing units (NPUs). These NPU chips ha…
Enabling Spatially Fine-Grained DVFS in Neural Processing Units for Energy-Efficient LLM Serving
Yuqi Xue, Jerry Wu, Corey Yu +1
As neural processing units (NPUs) evolve rapidly to accommodate the ever-increasing compute demand of large language models (LLMs), their power consumption is becoming a limiting f…
Exploring the Efficiency of 3D-Stacked AI Chip Architecture for LLM Inference with VOXEL
Yiqi Liu, Noelle Crawford, Michael Wang +2
To overcome the well-known memory bottleneck of AI chips, 3D-stacked architectures that employ advanced packaging technology with high-density through-silicon vias (TSVs) pins have…
ReGate: Enabling Power Gating in Neural Processing Units
Yuqi Xue, Jian Huang
The energy efficiency of neural processing units (NPU) is playing a critical role in developing sustainable data centers. Our study with different generations of NPU chips reveals…
ELK: Exploring the Efficiency of Inter-core Connected AI Chips with Deep Learning Compiler Techniques
Yiqi Liu, Yuqi Xue, Noelle Crawford +2
To meet the increasing demand of deep learning (DL) models, AI chips are employing both off-chip memory (e.g., HBM) and high-bandwidth low-latency interconnect for direct inter-cor…
Hardware-Assisted Virtualization of Neural Processing Units for Cloud Platforms
Yuqi Xue, Yiqi Liu, Lifeng Nai +1
Cloud platforms today have been deploying hardware accelerators like neural processing units (NPUs) for powering machine learning (ML) inference services. To maximize the resource…
G10: Enabling An Efficient Unified GPU Memory and Storage Architecture with Smart Tensor Migrations
Haoyang Zhang, Yirui Eric Zhou, Yuqi Xue +2
To break the GPU memory wall for scaling deep learning workloads, a variety of architecture and system techniques have been proposed recently. Their typical approaches include memo…