From the 1 of 9 linked papers with an AI index.
9 papers
HBF Sucks! A Full-Stack Characterization of High-Bandwidth Flash for KV-Centric LLM Serving
Zhuoran Li, Zhuohang Bian, Xin Huang +3
A faster storage device should make serving faster. We find the opposite. High-Bandwidth Flash (HBF) stacks NAND behind a wide, package-local interface, promising flash-scale capac…
CODA: Algorithm-Hardware Co-design for Edge Video Diffusion via NMP-Enabled Compute-Cache Operator Disaggregation
Yuanpeng Zhang, YuXuan Wu, Yitong Xiao +6
The paper introduces CODA, a hardware-software co-designed architecture that separates compute and cache operations for edge video diffusion models, using near‑memory processing to…
Isolation-aware Scheduling Framework for DNN-based End-to-End Autonomous Driving System on Tile-based Accelerators
Chenguang Zhang, Yuanpeng Zhang, Chenhao Xue +3
Level-4+ autonomous driving systems (ADS) must run dozens of heterogeneous deep neural networks (DNNs) as end-to-end (E2E) pipelines under a strict latency constraint (<=100 ms), e…
MoE-Hub: Taming Software Complexity for Seamless MoE Overlap with Hardware-Accelerated Communication on Multi-GPU Systems
Zhuoshan Zhou, Chen Zhang, Shuyi Zhang +10
The Mixture-of-Experts (MoE) architecture is crucial for scaling large language models, but its scalability is severely limited by inter-GPU communication bottlenecks in multi-GPU…
TokenStack: A Heterogeneous HBM-PIM Architecture and Runtime for Efficient LLM Inference
Zhuoran Li, Zhuohang Bian, Zihao Huang +5
Large language model (LLM) serving is now limited by the key-value (KV) cache. During decode, each new token rereads prior KV state, so attention becomes a bandwidth- and capacity-…
Towards Compute-Aware In-Switch Computing for LLMs Tensor-Parallelism on Multi-GPU Systems
Chen Zhang, Qijun Zhang, Zhuoshan Zhou +10
Tensor parallelism (TP) in large-scale LLM inference and training introduces frequent collective operations that dominate inter-GPU communication. While in-switch computing, exempl…