Compact Recurrent Transformer with Persistent Memory

Edison Mucllari, Zachary Daniels, David Zhang +1

The Transformer architecture has shown significant success in many language processing and visual tasks. However, the method faces challenges in efficiently scaling to long sequenc…