most citedLaMPP: Language Models as Probabilistic Priors for Perception and Action

4 citations · 4 across the 4 of their papers we have counts for

collaborators
Showing cs.CLShow all

2 papers · 1 filter

cs.CL2023

Pushdown Layers: Encoding Recursive Structure in Transformer Language Models

Shikhar Murty, Pratyusha Sharma, Jacob Andreas +1

Recursion is a prominent feature of human language, and fundamentally challenging for self-attention due to the lack of an explicit recursive-state tracking mechanism. Consequently…

cs.CL2023

Grokking of Hierarchical Structure in Vanilla Transformers

Shikhar Murty, Pratyusha Sharma, Jacob Andreas +1

For humans, language production and comprehension is sensitive to the hierarchical structure of sentences. In natural language processing, past work has questioned how effectively…