Showing cs.CLShow all
3 papers · 1 filter
cs.CL2025
MSWA: Refining Local Attention with Multi-ScaleWindow Attention
Yixing Xu, Shivank Nag, Dong Li +2
Transformer-based LLMs have achieved exceptional performance across a wide range of NLP tasks. However, the standard self-attention mechanism suffers from quadratic time complexity…
cs.CL2024
FTP: A Fine-grained Token-wise Pruner for Large Language Models via Token Routing
Zekai Li, Jintu Zheng, Ji Liu +9
Recently, large language models (LLMs) have demonstrated superior performance across various tasks by adhering to scaling laws, which significantly increase model size. However, th…
cs.CL2024
Enhancing One-shot Pruned Pre-trained Language Models through Sparse-Dense-Sparse Mechanism
Guanchen Li, Xiandong Zhao, Lian Liu +6
Pre-trained language models (PLMs) are engineered to be robust in contextual understanding and exhibit outstanding performance in various natural language processing tasks. However…