4 papers
TASP: Topology-aware Sequence Parallelism
Yida Wang, Ke Hong, Xiuhong Li +4
Long-context large language models (LLMs) face constraints due to the quadratic complexity of the self-attention mechanism. The mainstream sequence parallelism (SP) method, Ring At…
Efficient and Adaptable Overlapping for Computation and Communication via Signaling and Reordering
Ke Hong, Xiuhong Li, Minxu Liu +9
Generative models have achieved remarkable success across various applications, driving the demand for multi-GPU computing. Inter-GPU communication becomes a bottleneck in multi-GP…
PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and Quantized Attention in Visual Generation Models
Tianchen Zhao, Ke Hong, Xinhao Yang +8
In visual generation, the quadratic complexity of attention mechanisms results in high memory and computational costs, especially for longer token sequences required in high-resolu…
semi-PD: Towards Efficient LLM Serving via Phase-Wise Disaggregated Computation and Unified Storage
Ke Hong, Lufang Chen, Zhong Wang +9
Existing large language model (LLM) serving systems fall into two categories: 1) a unified system where prefill phase and decode phase are co-located on the same GPU, sharing the u…