Exact Sequence Interpolation with Transformers

Albert Alcalde, Giovanni Fantuzzi, Enrique Zuazua

We prove that transformers can exactly interpolate datasets of finite input sequences in ${\mathbb{R}}^d$, $d\ge 2$, with corresponding output sequences of smaller or equal length.…