3 papers
cs.LG2025
SOLE: Hardware-Software Co-design of Softmax and LayerNorm for Efficient Transformer Inference
Wenxun Wang, Shuchang Zhou, Wenyu Sun +2
Transformers have shown remarkable performance in both natural language processing (NLP) and computer vision (CV) tasks. However, their real-time inference speed and efficiency are…
cs.LG2025
TASP: Topology-aware Sequence Parallelism
Yida Wang, Ke Hong, Xiuhong Li +4
Long-context large language models (LLMs) face constraints due to the quadratic complexity of the self-attention mechanism. The mainstream sequence parallelism (SP) method, Ring At…
cs.LG2022
Block-Wise Dynamic-Precision Neural Network Training Acceleration via Online Quantization Sensitivity Analytics
Ruoyang Liu, Chenhan Wei, Yixiong Yang +3
Data quantization is an effective method to accelerate neural network training and reduce power consumption. However, it is challenging to perform low-bit quantized training: the c…