most citedARWKV: Pretrain is not what we need, an RNN-Attention-Based Language Model Born from Transformer

1 citations · 1 across the 10 of their papers we have counts for

collaborators
Showing cs.LGShow all

3 papers · 1 filter

cs.LG2026

Belief-State RWKV for Reinforcement Learning under Partial Observability

Liu Xiao

We propose a stronger formulation of RL on top of RWKV-style recurrent sequence models, in which the fixed-size recurrent state is explicitly interpreted as a belief state rather t…

cs.LG2025

Millions of States: Designing a Scalable MoE Architecture with RWKV-7 Meta-learner

Liu Xiao, Li Zhiyuan, Lin Yueyu

State-based sequence models like RWKV-7 offer a compelling alternative to Transformer architectures, achieving linear complexity while demonstrating greater expressive power in sho…

cs.LG2025

BlackGoose Rimer: Harnessing RWKV-7 as a Simple yet Superior Replacement for Transformers in Large-Scale Time Series Modeling

Li weile, Liu Xiao

Time series models face significant challenges in scaling to handle large and complex datasets, akin to the scaling achieved by large language models (LLMs). The unique characteris…