Cached Transformers: Improving Transformers with Differentiable Memory Cache

Zhaoyang Zhang, Wenqi Shao, Yixiao Ge +3

This work introduces a new Transformer model called Cached Transformer, which uses Gated Recurrent Cached (GRC) attention to extend the self-attention mechanism with a differentiab…