Universal interpolation for deep residual self-attention networks

Sibylle Marcotte, Joan Bruna

Universal approximation is a necessary qualitative property of learning architectures to benefit from scaling laws. While it is generically verified on a variety of neural architec…