2 citations · 2 across the 15 of their papers we have counts for
15 papers · 1 filter
Compositional Multilingual and Behavioral Attribute Steering
Hyun Gu Kang, Daniil Gurgurov, Tanja Baeumel +2
This study examines the compositionality of steering vectors for language and behavioral control in large language models. Focusing on language, jailbreak, and conciseness, we inve…
When Tokenization is Secretly Output Supervision
Tanja Baeumel, Josef van Genabith, Simon Ostermann
Tokenization in language models is treated by default as an input preprocessing decision. We argue that this framing is incomplete: in autoregressive models, tokenizer granularity…
Separating Syntax from Language: A Mechanistic Account of Translation in Multilingual LLMs
Mikhail Sonkin, Tanja Baeumel, Daniil Gurgurov +2
Multilingual large language models (mLLMs) achieve strong performance in machine translation, yet our understanding of the mechanisms by which they transform representations from o…
Multilingual Steering by Design: Multilingual Sparse Autoencoders and Principled Layer Selection
Yusser Al Ghussin, Daniil Gurgurov, Tanja Baeumel +3
Sparse autoencoders (SAEs) enable feature-level mechanistic interpretability and activation steering in large language models (LLMs), but SAE-based language control remains unrelia…
Judge Circuits Explain Format-Induced Inconsistency in LLM-as-a-Judge
Nils Feldhus, Tanja Baeumel, Elena Golimblevskaia +10
LLM-as-a-judge has become the dominant paradigm for grading model outputs at scale, yet the same model assigns systematically different scores when its output format changes (e.g.,…
Disentangling Mathematical Reasoning in LLMs: A Methodological Investigation of Internal Mechanisms
Tanja Baeumel, Josef van Genabith, Simon Ostermann
Large language models (LLMs) have demonstrated impressive capabilities, yet their internal mechanisms for handling reasoning-intensive tasks remain underexplored. To advance the un…
From Weights to Activations: Is Steering the Next Frontier of Adaptation?
Simon Ostermann, Daniil Gurgurov, Tanja Baeumel +4
Post-training adaptation of language models is commonly achieved through parameter updates or input-based methods such as fine-tuning, parameter-efficient adaptation, and prompting…
CLaS-Bench: A Cross-Lingual Alignment and Steering Benchmark
Daniil Gurgurov, Yusser Al Ghussin, Tanja Baeumel +5
Understanding and controlling the behavior of large language models (LLMs) is an increasingly important topic in multilingual NLP. Beyond prompting or fine-tuning, , i.e.,~manipula…
Modular Arithmetic: Language Models Solve Math Digit by Digit
Tanja Baeumel, Daniil Gurgurov, Yusser al Ghussin +2
While recent work has begun to uncover the internal strategies that Large Language Models (LLMs) employ for simple arithmetic tasks, a unified understanding of their underlying mec…
What Language(s) Does Aya-23 Think In? How Multilinguality Affects Internal Language Representations
Katharina Trinley, Toshiki Nakai, Tatiana Anikina +1
Large language models (LLMs) excel at multilingual tasks, yet their internal language processing remains poorly understood. We analyze how Aya-23-8B, a decoder-only LLM trained on…
Language Arithmetics: Towards Systematic Language Neuron Identification and Manipulation
Daniil Gurgurov, Katharina Trinley, Yusser Al Ghussin +3
Large language models (LLMs) exhibit strong multilingual abilities, yet the neural mechanisms behind language-specific processing remain unclear. We analyze language-specific neuro…
The Lookahead Limitation: Why Multi-Operand Addition is Hard for LLMs
Tanja Baeumel, Josef van Genabith, Simon Ostermann
Autoregressive large language models (LLMs) exhibit impressive performance across various tasks but struggle with simple arithmetic, such as addition of two or more operands. We sh…
Multilingual Large Language Models and Curse of Multilinguality
Daniil Gurgurov, Tanja Bäumel, Tatiana Anikina
Multilingual Large Language Models (LLMs) have gained large popularity among Natural Language Processing (NLP) researchers and practitioners. These models, trained on huge datasets…
Where exactly does contextualization in a PLM happen?
Soniya Vijayakumar, Tanja Bäumel, Simon Ostermann +1
Pre-trained Language Models (PLMs) have shown to be consistently successful in a plethora of NLP tasks due to their ability to learn contextualized representations of words (Ethaya…
Investigating the Encoding of Words in BERT's Neurons using Feature Textualization
Tanja Baeumel, Soniya Vijayakumar, Josef van Genabith +2
Pretrained language models (PLMs) form the basis of most state-of-the-art NLP technologies. Nevertheless, they are essentially black boxes: Humans do not have a clear understanding…