17 citations · 40 across the 4 of their papers we have counts for
4 papers
Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence
Bo Peng, Daniel Goldstein, Quentin Anthony +27
We present Eagle (RWKV-5) and Finch (RWKV-6), sequence models improving upon the RWKV (RWKV-4) architecture. Our architectural design advancements include multi-headed matrix-value…
RWKV: Reinventing RNNs for the Transformer Era
Bo Peng, Eric Alcaide, Quentin Anthony +31
Transformers have revolutionized almost all natural language processing (NLP) tasks but suffer from memory and computational complexity that scales quadratically with sequence leng…
Improving Graph Property Prediction with Generalized Readout Functions
Eric Alcaide
Graph property prediction is drawing increasing attention in the recent years due to the fact that graphs are one of the most general data structures since they can contain an arbi…
E-swish: Adjusting Activations to Different Network Depths
Eric Alcaide
Activation functions have a notorious impact on neural networks on both training and testing the models against the desired problem. Currently, the most used activation function is…