7 citations · 7 across the 4 of their papers we have counts for
1 paper · 1 filter
Haowen Hou, Zhiyi Huang, Kaifeng Tan +2
In this paper, we introduce RWKV-X, a novel hybrid architecture that combines the efficiency of RWKV for short-range modeling with a sparse attention mechanism designed to capture…