Residual Shuffle-Exchange Networks for Fast Processing of Long Sequences

Andis Draguns, Emīls Ozoliņš, Agris Šostaks +2

Attention is a commonly used mechanism in sequence processing, but it is of O(n^2) complexity which prevents its application to long sequences. The recently introduced neural Shuff…