On the Generalization Ability of Retrieval-Enhanced Transformers

Tobias Norlund, Ehsan Doostmohammadi, Richard Johansson +1

Recent work on the Retrieval-Enhanced Transformer (RETRO) model has shown that off-loading memory from trainable weights to a retrieval database can significantly improve language…