Trees in transformers: a theoretical analysis of the Transformer's ability to represent trees

Qi He, João Sedoc, Jordan Rodu

Transformer networks are the de facto standard architecture in natural language processing. To date, there are no theoretical analyses of the Transformer's ability to capture tree…