4 citations · 15 across the 21 of their papers we have counts for
Showing 2025Show all
3 papers · 1 filter
cs.CL2025
StateX: Enhancing RNN Recall via Post-training State Expansion
Xingyu Shen, Yingfa Chen, Zhen Leng Thai +3
Recurrent neural networks (RNNs), such as linear attention and state-space models, have gained popularity due to their constant per-token complexity when processing long contexts.…
cs.LG2025
BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity
Chenyang Song, Weilin Zhao, Xu Han +5
To alleviate the computational burden of large language models (LLMs), architectures with activation sparsity, represented by mixture-of-experts (MoE), have attracted increasing at…
cs.CL2025
Cost-Optimal Grouped-Query Attention for Long-Context Modeling
Yingfa Chen, Yutong Wu, Chenyang Song +5
Grouped-Query Attention (GQA) is a widely adopted strategy for reducing the computational cost of attention layers in large language models (LLMs). However, current GQA configurati…