RRWKV: Capturing Long-range Dependencies in RWKV

Leilei Wang

Owing to the impressive dot-product attention, the Transformers have been the dominant architectures in various natural language processing (NLP) tasks. Recently, the Receptance We…