Attention is All You Need Until You Need Retention

M. Murat Yaslioglu

Pretrained Transformers keep what they learned in their weights and lose what they observe once a session ends. The first version of this paper proposed a Retention Layer, a persis…