3 citations · 5 across the 3 of their papers we have counts for
3 papers
cs.CL2024★ 3 cited
Why Does the Effective Context Length of LLMs Fall Short?
Chenxin An, Jun Zhang, Ming Zhong +5
Advancements in distributed training and efficient attention mechanisms have significantly expanded the context window sizes of large language models (LLMs). However, recent work r…
cs.CL2024★ 2 cited
Training-Free Long-Context Scaling of Large Language Models
Chenxin An, Fei Huang, Jun Zhang +4
The ability of Large Language Models (LLMs) to process and generate coherent text is markedly weakened when the number of input tokens exceeds their pretraining length. Given the e…
cs.CL2023
Linear Attention via Orthogonal Memory
Jun Zhang, Shuyang Jiang, Jiangtao Feng +2
Efficient attentions have greatly improved the computational efficiency of Transformers. However, most existing linear attention mechanisms suffer from an \emph{efficiency degradat…