4 papers
LoopMTP: A looped transformer guided by latent multi-token prediction
Behzad Shomali, Markus Frey, David Berghaus +2
Looped transformers have emerged as a parameter-efficient alternative to scaling depth for strong reasoning. By reusing one stack of layers across iterations, they attain the e…
LLM Parameters for Math Across Languages: Shared or Separate?
Behzad Shomali, Luisa Victor, Tim Selbach +5
Large language models (LLMs) exhibit substantial cross-lingual variation in mathematical reasoning performance, but it remains unclear whether these differences reflect language-sp…
A Dual-Path Architecture for Scaling Compute and Capacity in LLMs
Markus Frey, Behzad Shomali, Joachim Koehler +1
Looped transformers apply a shared block multiple times and have emerged as a parameter-efficient route to scaling compute in language models. However, at fixed FLOPs a looped mode…
Adaptive Loops and Memory in Transformers: Think Harder or Know More?
Markus Frey, Behzad Shomali, Ali Hamza Bashir +3
Chain-of-thought (CoT) prompting enables reasoning in language models but requires explicit verbalization of intermediate steps. Looped transformers offer an alternative by iterati…