Grokking of Hierarchical Structure in Vanilla Transformers

Shikhar Murty, Pratyusha Sharma, Jacob Andreas +1

For humans, language production and comprehension is sensitive to the hierarchical structure of sentences. In natural language processing, past work has questioned how effectively…