6 papers
TileSight: A First-Principles Tile-Centric Analytical GPU Performance Model from Cores to Clusters
Zhiwen Mo, Yu Cheng, Lei Wang +12
Recent GPU programming frameworks such as Triton, TileLang, and CUDA Tile adopt tiles as first-class primitives, making tile-centric programming the prevailing approach for high-pe…
A Full-Stack Performance Evaluation Infrastructure for 3D-DRAM-based LLM Accelerators
Cong Li, Chenhao Xue, Yi Ren +11
Large language models (LLMs) exhibit memory-intensive behavior during decoding, making it a key bottleneck in LLM inference. To accelerate decoding execution, hybrid-bonding-based…
DeepStack: Facilitating Co-Design Exploration of 3D DRAM-Stacked Accelerators for Distributed LLM Inference
Zhiwen Mo, Guoyu Li, Hao Mark Chen +11
Advances in hybrid bonding and packaging have driven growing interest in 3D DRAM-stacked AI accelerators. As large language models (LLMs) scale to hundreds of billions or trillions…
SeerAttention-R: Sparse Attention Adaptation for Long Reasoning
Yizhao Gao, Shuming Guo, Shijie Cao +12
We introduce SeerAttention-R, a sparse attention framework specifically tailored for the long decoding of reasoning models. Extended from SeerAttention, SeerAttention-R retains the…
TileLang: A Composable Tiled Programming Model for AI Systems
Lei Wang, Yu Cheng, Yining Shi +8
Modern AI workloads rely heavily on optimized computing kernels for both training and inference. These AI kernels follow well-defined data-flow patterns, such as moving tiles betwe…
AttentionEngine: A Versatile Framework for Efficient Attention Mechanisms on Diverse Hardware Platforms
Feiyang Chen, Yu Cheng, Lei Wang +8
Transformers and large language models (LLMs) have revolutionized machine learning, with attention mechanisms at the core of their success. As the landscape of attention variants e…