4 citations · 9 across the 8 of their papers we have counts for
Showing 2025Show all
2 papers · 1 filter
cs.LG2025
BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsity
Chenyang Song, Weilin Zhao, Xu Han +5
To alleviate the computational burden of large language models (LLMs), architectures with activation sparsity, represented by mixture-of-experts (MoE), have attracted increasing at…
cs.CL2025
Cost-Optimal Grouped-Query Attention for Long-Context Modeling
Yingfa Chen, Yutong Wu, Chenyang Song +5
Grouped-Query Attention (GQA) is a widely adopted strategy for reducing the computational cost of attention layers in large language models (LLMs). However, current GQA configurati…