Transformers as Transducers

Lena Strobl, Dana Angluin, David Chiang +2

We study the sequence-to-sequence mapping capacity of transformers by relating them to finite transducers, and find that they can express surprisingly large classes of transduction…