Learning State-Tracking from Code Using Linear RNNs

Julien Siems, Riccardo Grazzi, Korbinian Pöppel +3

Over the last years, state-tracking tasks, particularly permutation composition, have become a testbed to understand the limits of sequence models architectures like Transformers a…