3 papers
cs.DC2026
CacheFlow: Efficient LLM Serving with 3D-Parallel KV Cache Restoration
Sean Nian, Jiahao Fang, Qilong Feng +2
KV cache restoration has emerged as a dominant bottleneck in serving long-context LLM workloads, including multi-turn conversations, retrieval-augmented generation, and agentic pip…
cs.AR2024
SOFA: A Compute-Memory Optimized Sparsity Accelerator via Cross-Stage Coordinated Tiling
Huizheng Wang, Jiahao Fang, Xinru Tang +9
Benefiting from the self-attention mechanism, Transformer models have attained impressive contextual comprehension capabilities for lengthy texts. The requirements of high-throughp…
cs.DC2024
PALM: A Efficient Performance Simulator for Tiled Accelerators with Large-scale Model Training
Jiahao Fang, Huizheng Wang, Qize Yang +5
Deep learning (DL) models are piquing high interest and scaling at an unprecedented rate. To this end, a handful of tiled accelerators have been proposed to support such large-scal…