Showing cs.FLShow all
2 papers · 1 filter
cs.FL2025
The Role of Logic and Automata in Understanding Transformers
Anthony W. Lin, Pablo Barcelo
The advent of transformers has in recent years led to powerful and revolutionary Large Language Models (LLMs). Despite this, our understanding on the capability of transformers is…
cs.FL2024
The Power of Hard Attention Transformers on Data Sequences: A Formal Language Theoretic Perspective
Pascal BergsträÃer, Chris Köcher, Anthony Widjaja Lin +1
Formal language theory has recently been successfully employed to unravel the power of transformer encoders. This setting is primarily applicable in Natural Language Processing (NL…