Theoretical Analysis of Hierarchical Language Recognition and Generation by Transformers without Positional Encoding

Daichi Hayakawa, Issei Sato

In this study, we provide constructive proof that Transformers can recognize and generate hierarchical language efficiently with respect to model size, even without the need for a…