3 papers
cs.LG2025
Post-Norm can Resharpen Attention
Pál Zsámboki, Benjamin Levi, David Ansel Josef Smith +4
Length Generalization is the essential capacity of autonomous agents to perform tasks in longer contexts than those encountered during training. To systematically study this feat,…
cs.LG2025
Do Attention Heads Compete or Cooperate during Counting?
Pál Zsámboki, Ádám Fraknói, Máté Gedeon +2
We present an in-depth mechanistic interpretability analysis of training small transformers on an elementary task, counting, which is a crucial deductive step in many algorithms. I…
math.AG2018
On the -stack of complexes over a scheme
Ajneet Dhillon, Pál Zsámboki
We study fppf descent for enhanced derived categories. We revisit the work of [HS] and [TV08] in a lax context. More precisely, we construct a Cartesian and coCartesian fibration $…