884 citations · 923 across the 3 of their papers we have counts for
3 papers · 1 filter
DualPath: Breaking the Storage Bandwidth Bottleneck in Agentic LLM Inference
Yongtong Wu, Shaoyuan Chen, Yinmin Zhong +10
The performance of multi-turn, agentic LLM inference is increasingly dominated by KV-Cache storage I/O rather than computation. In prevalent disaggregated architectures, loading th…
Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures
Chenggang Zhao, Chengqi Deng, Chong Ruan +12
The rapid scaling of large language models (LLMs) has unveiled critical limitations in current hardware architectures, including constraints in memory capacity, computational effic…
Fire-Flyer AI-HPC: A Cost-Effective Software-Hardware Co-Design for Deep Learning
Wei An, Xiao Bi, Guanting Chen +49
The rapid progress in Deep Learning (DL) and Large Language Models (LLMs) has exponentially increased demands of computational power and bandwidth. This, combined with the high cos…