most citedARWKV: Pretrain is not what we need, an RNN-Attention-Based Language Model Born from Transformer

1 citations · 1 across the 5 of their papers we have counts for

collaborators
Showing cs.LGShow all

1 paper · 1 filter

cs.LG2025

Millions of States: Designing a Scalable MoE Architecture with RWKV-7 Meta-learner

Liu Xiao, Li Zhiyuan, Lin Yueyu

State-based sequence models like RWKV-7 offer a compelling alternative to Transformer architectures, achieving linear complexity while demonstrating greater expressive power in sho…