29 citations · 35 across the 36 of their papers we have counts for
3 papers · 1 filter
KV-RM: Regularizing KV-Cache Movement for Static-Graph LLM Serving
Zhiqing Zhong, Zhijing Ye, Jian Zhang +3
Static-graph LLM decoders provide predictable launches, fixed tensor shapes, and low submission overhead, but online decoding exposes highly irregular KV-cache behavior: request le…
DABench-LLM: Standardized and In-Depth Benchmarking of Post-Moore Dataflow AI Accelerators for LLMs
Ziyu Hu, Zhiqing Zhong, Weijian Zheng +6
The exponential growth of large language models has outpaced the capabilities of traditional CPU and GPU architectures due to the slowdown of Moore's Law. Dataflow AI accelerators…
GFormer: Accelerating Large Language Models with Optimized Transformers on Gaudi Processors
Chengming Zhang, Xinheng Ding, Baixi Sun +4
Heterogeneous hardware like Gaudi processor has been developed to enhance computations, especially matrix operations for Transformer-based large language models (LLMs) for generati…