3 papers
cs.FL2026
Algebraic Decomposition Theory for Transformer Length Generalization
Andy Yang, Blerta Veseli, Corentin Barloy +5
Transformer-based language models are known to sometimes generalize to sequences longer than seen during training, but we lack a precise characterization of which tasks admit lengt…
cs.LG2025
Can LLMs subtract numbers?
Mayank Jobanputra, Nils Philipp Walter, Maitrey Mehta +7
We present a systematic study of subtraction in large language models (LLMs). While prior benchmarks emphasize addition and multiplication, subtraction has received comparatively l…
cs.CL2025
Positional Biases Shift as Inputs Approach Context Window Limits
Blerta Veseli, Julian Chibane, Mariya Toneva +1
Large Language Models (LLMs) often struggle to use information across long inputs effectively. Prior work has identified positional biases, such as the Lost in the Middle (LiM) eff…