5 papers
LoopMTP: A looped transformer guided by latent multi-token prediction
Behzad Shomali, Markus Frey, David Berghaus +2
Looped transformers have emerged as a parameter-efficient alternative to scaling depth for strong reasoning. By reusing one stack of layers across iterations, they attain the e…
HiFi-LLP: High-Fidelity, Low-Cost Latency Predictors with Confidence for Robust HW-NAS
Shambhavi Balamuthu Sampath, Behzad Shomali, Nael Fasfous +7
With deep neural networks (DNNs) increasingly deployed on edge devices, hardware (HW)-aware optimization techniques--such as HW-aware compression and HW-aware neural architecture s…
LLM Parameters for Math Across Languages: Shared or Separate?
Behzad Shomali, Luisa Victor, Tim Selbach +5
Large language models (LLMs) exhibit substantial cross-lingual variation in mathematical reasoning performance, but it remains unclear whether these differences reflect language-sp…
Adaptive Loops and Memory in Transformers: Think Harder or Know More?
Markus Frey, Behzad Shomali, Ali Hamza Bashir +3
Chain-of-thought (CoT) prompting enables reasoning in language models but requires explicit verbalization of intermediate steps. Looped transformers offer an alternative by iterati…
Is continuous CoT better suited for multi-lingual reasoning?
Ali Hamza Bashir, Behzad Shomali, Markus Frey +3
We investigate whether performing reasoning in a continuous latent space leads to more robust multilingual capabilities. We compare Continuous Chain-of-Thought (using the CODI fram…