Distil-xLSTM: Learning Attention Mechanisms through Recurrent Structures

Abdoul Majid O. Thiombiano, Brahim Hnich, Ali Ben Mrad +1

The current era of Natural Language Processing (NLP) is dominated by Transformer models. However, novel architectures relying on recurrent mechanisms, such as xLSTM and Mamba, have…