Length-Independent State Tracking Under a Parallel Scan

Julien Brandoit, Arthur Fyon, Thomas Braipson +5

Learning robust and scalable finite-state tracking is fundamental to sequence processing. While linear recurrent neural networks (RNNs), linear attention, and state space models en…