An All-MLP Sequence Modeling Architecture That Excels at Copying

Chenwei Cui, Zehao Yan, Gedeon Muhawenayo +1

Recent work demonstrated Transformers' ability to efficiently copy strings of exponential sizes, distinguishing them from other architectures. We present the Causal Relation Networ…