On the Expressive Power of Transformers for Maxout Networks and Continuous Piecewise Linear Functions

Linyan Gu, Lihua Yang, Feng Zhou

Transformer networks have achieved remarkable empirical success across a wide range of applications, yet their theoretical expressive power remains insufficiently understood. In th…