3 citations · 3 across the 7 of their papers we have counts for
Showing 2025Show all
3 papers · 1 filter
cs.LG2025
Flash Multi-Head Feed-Forward Network
Minshen Zhang, Xiang Hu, Jianguo Li +2
We explore Multi-Head FFN (MH-FFN) as a replacement of FFN in the Transformer architecture, motivated by the structural similarity between single-head attention and FFN. While mult…
cs.CL2025
Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models
Xiang Hu, Zhanchao Zhou, Ruiqi Liang +3
This work explores the challenge of building ``Machines that Can Remember'', framing long-term memory as the problem of efficient ultra-long context modeling. We argue that this re…
cs.CL2025
Understanding and Improving Length Generalization in Hierarchical Sparse Attention Models
Jiaqi Leng, Xiang Hu, Junxiong Wang +3
Effectively processing long contexts is a critical challenge for language models. While standard Transformers are limited by quadratic complexity and poor length extrapolation, alt…