most citedARWKV: Pretrain is not what we need, an RNN-Attention-Based Language Model Born from Transformer

1 citations · 1 across the 5 of their papers we have counts for

collaborators
Showing cs.CLShow all

2 papers · 1 filter

cs.CL2025

WuNeng: Hybrid State with Attention

Liu Xiao, Li Zhiyuan, Lin Yueyu

The WuNeng architecture introduces a novel approach to enhancing the expressivity and power of large language models by integrating recurrent neural network (RNN)-based RWKV-7 with…

cs.CL2025★ 1 cited

ARWKV: Pretrain is not what we need, an RNN-Attention-Based Language Model Born from Transformer

Lin Yueyu, Li Zhiyuan, Peter Yue +1

As is known, hybrid quadratic and subquadratic attention models in multi-head architectures have surpassed both Transformer and Linear RNN models , with these works primarily focus…