Showing cs.LGShow all
2 papers · 1 filter
cs.LG2025
DistrAttention: An Efficient and Flexible Self-Attention Mechanism on Modern GPUs
Haolin Jin, Mengbai Xiao, Yuan Yuan +4
The Transformer architecture has revolutionized deep learning, delivering the state-of-the-art performance in areas such as natural language processing, computer vision, and time s…
cs.LG2025
SpecPipe: Accelerating Pipeline Parallelism-based LLM Inference with Speculative Decoding
Haofei Yin, Mengbai Xiao, Tinghong Li +3
The demand for large language model inference is rapidly increasing. Pipeline parallelism offers a cost-effective deployment strategy for distributed inference but suffers from hig…