5 papers
LLM Parameters for Math Across Languages: Shared or Separate?
Behzad Shomali, Luisa Victor, Tim Selbach +5
Large language models (LLMs) exhibit substantial cross-lingual variation in mathematical reasoning performance, but it remains unclear whether these differences reflect language-sp…
Adaptive Loops and Memory in Transformers: Think Harder or Know More?
Markus Frey, Behzad Shomali, Ali Hamza Bashir +3
Chain-of-thought (CoT) prompting enables reasoning in language models but requires explicit verbalization of intermediate steps. Looped transformers offer an alternative by iterati…
Is continuous CoT better suited for multi-lingual reasoning?
Ali Hamza Bashir, Behzad Shomali, Markus Frey +3
We investigate whether performing reasoning in a continuous latent space leads to more robust multilingual capabilities. We compare Continuous Chain-of-Thought (using the CODI fram…
Domain-Adaptation through Synthetic Data: Fine-Tuning Large Language Models for German Law
Ali Hamza Bashir, Muhammad Rehan Khalid, Kostadin Cvejoski +7
Large language models (LLMs) often struggle in specialized domains such as legal reasoning due to limited expert knowledge, resulting in factually incorrect outputs or hallucinatio…
Once Upon a Time: Interactive Learning for Storytelling with Small Language Models
Jonas Mayer Martins, Ali Hamza Bashir, Muhammad Rehan Khalid +1
Children efficiently acquire language not just by listening, but by interacting with others in their social environment. Conversely, large language models are typically trained wit…