Approximation of Permutation Invariant Polynomials by Transformers: Efficient Construction in Column-Size

Naoki Takeshita, Masaaki Imaizumi

Transformers are a type of neural network that have demonstrated remarkable performance across various domains, particularly in natural language processing tasks. Motivated by this…