Interdomain Attention: Beyond Token-Level Key-Value Memory

Naoki Kiyohara, Harrison Bo Hua Zhu, Riccardo El Hassanin +4

Softmax attention-based transformers and recurrent memory-based models (e.g., state space models (SSMs)) form the foundation of modern sequence models. Hybrid architectures aim to…