Myosotis: structured computation for attention like layer

Evgenii Egorov, Hanno Ackermann, Markus Nagel +1

Attention layers apply a sequence-to-sequence mapping whose parameters depend on the pairwise interactions of the input elements. However, without any structural assumptions, memor…