2 papers
cs.LG2026
Improved state mixing in higher-order and block diagonal linear recurrent networks
Igor Dubinin, Antonio Orvieto, Felix Effenberger
Linear recurrent networks (LRNNs) and linear state space models (SSMs) promise computational and memory efficiency on long-sequence modeling tasks, yet their diagonal state transit…
cs.LG2023
Fading memory as inductive bias in residual recurrent networks
Igor Dubinin, Felix Effenberger
Residual connections have been proposed as an architecture-based inductive bias to mitigate the problem of exploding and vanishing gradients and increased task performance in both…