Showing cs.LGShow all
3 papers · 1 filter
cs.LG2025
Concise One-Layer Transformers Can Do Function Evaluation (Sometimes)
Lena Strobl, Dana Angluin, Robert Frank
While transformers have proven enormously successful in a range of tasks, their fundamental properties as models of computation are not well understood. This paper contributes to t…
cs.LG2024
Simulating Hard Attention Using Soft Attention
Andy Yang, Lena Strobl, David Chiang +1
We study conditions under which transformers using soft attention can simulate hard attention, that is, effectively focus all attention on a subset of positions. First, we examine…
cs.LG2023
What Formal Languages Can Transformers Express? A Survey
Lena Strobl, William Merrill, Gail Weiss +2
As transformers have gained prominence in natural language processing, some researchers have investigated theoretically what problems they can and cannot solve, by treating problem…