Dynamic Compression in Recurrent Networks

Jyothish Pari, Ryan Bahlous-Boldi, Pulkit Agrawal

Recurrent models process long contexts efficiently by compressing their history into a fixed-size state, but modern architectures typically do so in a single causal pass over the s…