Showing cs.LGShow all
3 papers · 1 filter
cs.LG2026
Length Generalization Bounds for Transformers
Andy Yang, Pascal BergsträÃer, Georg Zetzsche +2
Length generalization is a key property of a learning algorithm that enables it to make correct predictions on inputs of any length, given finite training data. To provide such a g…
cs.LG2025
The Transformer Cookbook
Andy Yang, Christopher Watson, Anton Xue +6
We present the transformer cookbook: a collection of techniques for directly encoding algorithms into a transformer's parameters. This work addresses the steep learning curve of su…
cs.LG2025
Simulating Hard Attention Using Soft Attention
Andy Yang, Lena Strobl, David Chiang +1
We study conditions under which transformers using soft attention can simulate hard attention, that is, effectively focus all attention on a subset of positions. First, we examine…