15 citations · 15 across the 4 of their papers we have counts for
Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
ConceptMoE: Adaptive Token-to-Concept Compression for Implicit Compute Allocation
Zihao Huang, Jundong Zhou, Xingwei Qu +2
Large language models allocate uniform computation across all tokens, ignoring that some sequences are trivially predictable while others require deep reasoning. We introduce Conce…
cs.LG2024★ 15 cited
Learning to (Learn at Test Time): RNNs with Expressive Hidden States
Yu Sun, Xinhao Li, Karan Dalal +9
Self-attention performs well in long context but has quadratic complexity. Existing RNN layers have linear complexity, but their performance in long context is limited by the expre…