2 citations · 2 across the 2 of their papers we have counts for
Showing cs.FLShow all
2 papers · 1 filter
cs.FL2025
The Role of Logic and Automata in Understanding Transformers
Anthony W. Lin, Pablo Barcelo
The advent of transformers has in recent years led to powerful and revolutionary Large Language Models (LLMs). Despite this, our understanding on the capability of transformers is…
cs.FL2023★ 2 cited
Logical Languages Accepted by Transformer Encoders with Hard Attention
Pablo Barcelo, Alexander Kozachinskiy, Anthony Widjaja Lin +1
We contribute to the study of formal languages that can be recognized by transformer encoders. We focus on two self-attention mechanisms: (1) UHAT (Unique Hard Attention Transforme…