4 papers · 1 filter
State-Space Modeling in Long Sequence Processing: A Survey on Recurrence in the Transformer Era
Matteo Tiezzi, Michele Casoni, Alessandro Betti +2
Effectively learning from sequential data is a longstanding goal of Artificial Intelligence, especially in the case of long sequences. From the dawn of Machine Learning, several re…
Generative System Dynamics in Recurrent Neural Networks
Michele Casoni, Tommaso Guidi, Alessandro Betti +2
In this study, we investigate the continuous time dynamics of Recurrent Neural Networks (RNNs), focusing on systems with nonlinear activation functions. The objective of this work…
A Unified Framework for Neural Computation and Learning Over Time
Stefano Melacci, Alessandro Betti, Michele Casoni +3
This paper proposes Hamiltonian Learning, a novel unified framework for learning with neural networks "over time", i.e., from a possibly infinite stream of data, in an online manne…
Dynamic Decoupling of Placid Terminal Attractor-based Gradient Descent Algorithm
Jinwei Zhao, Marco Gori, Alessandro Betti +4
Gradient descent (GD) and stochastic gradient descent (SGD) have been widely used in a large number of application domains. Therefore, understanding the dynamics of GD and improvin…