2 papers
cs.LG2025
SOLE: Hardware-Software Co-design of Softmax and LayerNorm for Efficient Transformer Inference
Wenxun Wang, Shuchang Zhou, Wenyu Sun +2
Transformers have shown remarkable performance in both natural language processing (NLP) and computer vision (CV) tasks. However, their real-time inference speed and efficiency are…
cs.LG2025
TASP: Topology-aware Sequence Parallelism
Yida Wang, Ke Hong, Xiuhong Li +4
Long-context large language models (LLMs) face constraints due to the quadratic complexity of the self-attention mechanism. The mainstream sequence parallelism (SP) method, Ring At…