4 papers
PASCAL: A Phase-Aware Shared-Cache Model for Parallel Scans
Zhongchun Zhou, Chengtao Lai, Songtao Mao
In modern AI Accelerators and GPGPUs, many concurrent cores repeatedly access the same shared data. This pattern occurs in attention, where different query tiles share the same K/V…
Sim-FA: A GPGPU Simulator Framework for Fine-Grained Asynchronous Pipeline Analysis
Zhongchun Zhou, Yuhang Gu, Chengtao Lai +4
To efficiently support Large Language Models (LLMs), modern GPGPU architectures have introduced new features and programming paradigms, such as warp specialization. These features…
DCO: Dynamic Cache Orchestration for LLM Accelerators through Predictive Management
Zhongchun Zhou, Chengtao Lai, Yuhang Gu +1
The rapid adoption of large language models (LLMs) is pushing AI accelerators toward increasingly powerful and specialized designs. Instead of further complicating software develop…
LLaMCAT: Optimizing Large Language Model Inference with Cache Arbitration and Throttling
Zhongchun Zhou, Chengtao Lai, Wei Zhang
Large Language Models (LLMs) have achieved unprecedented success across various applications, but their substantial memory requirements pose significant challenges to current memor…