Attention is All You Need Until You Need Retention

M. Murat Yaslioglu

This work introduces a novel Retention Layer mechanism for Transformer based architectures, addressing their inherent lack of intrinsic retention capabilities. Unlike human cogniti…