5 papers
Post-Norm can Resharpen Attention
Pál Zsámboki, Benjamin Levi, David Ansel Josef Smith +4
Length Generalization is the essential capacity of autonomous agents to perform tasks in longer contexts than those encountered during training. To systematically study this feat,…
Commutative classifying space for simplicial groups
Cihan Okay, Pál Zsámboki
In this paper, we introduce a simplicial analog of classifying spaces for commutativity which classify principal bundles with commutativity structure on their transition functions.…
Diverse beam search to find densest-known planar unit distance graphs
Peter Engel, Owen Hammond-Lee, Yiheng Su +2
This paper addresses the problem of determining the maximum number of edges in a unit distance graph (UDG) of vertices using computer search. An unsolved problem of Paul ErdÅs…
The fractional chromatic number of the plane is at least 4
Máté Matolcsi, Imre Z. Ruzsa, Dániel Varga +1
We prove that the fractional chromatic number of the unit distance graph of the Euclidean plane is greater than or equal to . Interestingly, however, we cann…
Do Attention Heads Compete or Cooperate during Counting?
Pál Zsámboki, Ãdám Fraknói, Máté Gedeon +2
We present an in-depth mechanistic interpretability analysis of training small transformers on an elementary task, counting, which is a crucial deductive step in many algorithms. I…