12 papers
Detecting Hidden Behaviors in LLMs via Activation-matched Finetuning
Robin Haselhorst, Lucie Flek, Florian Mai
Large language models can hide hidden behaviors that activate only under narrow conditions, such as backdoor triggers, sleeper-agent deployment cues, sandbagging, or topic-conditio…
Data Attribution of Emergent Misalignment with Persona Features
Clemens Vetter, David Kaczér, Lucie Flek +1
Emergent misalignment (EM) is the phenomenon where fine-tuning a language model on a narrow task leads to harmful behavior in unrelated domains. A leading mechanistic account attri…
Reasoning Primitives in Hybrid and Non-Hybrid LLMs: Do Architectural Differences Yield Advantages in State-Tracking and Recall?
Shivam Rawat, Lucie Flek, Florian Mai +1
Reasoning in large language models is often discussed as a single capability, but some of its gains may stem from simpler underlying operations. We examine two such primitives, rec…
Transfer Learning Across Fast- and Full-Simulation Domains in High-Energy Physics
Matthias Schott, Lucie Flek
Machine-learning models in high-energy physics are often trained on simulated data, where fully simulated samples are computationally expensive while fast simulation provides large…
Uncovering Hidden Systematics in Neural Network Models for High Energy Physics
Lucie Flek, Philipp Alexander Jungs, Akbar Karimi +6
Neural networks (NNs) are inherently multidimensional classifiers that learn complex, non-linear relationships among input observables. While their flexibility enables unprecedente…
Learning Minimal-Deviation Corrections for Multi-Dimensional Mismodelling in HEP Simulations
Matthias Schott, Lucie Flek
Accurate Monte Carlo (MC) modelling in high-energy physics is challenging, particularly in complex scenarios where simulations fail to reproduce observed data. In practice, experim…
Tucano 2 Cool: Better Open Source LLMs for Portuguese
Nicholas Kluge Corrêa, Aniket Sen, Shiza Fatimah +4
We present Tucano 2, a fully open suite of large language models (LLMs) with 0.5-3.7 billion parameters, designed to address certain gaps in open-source development for Portuguese…
Raising Bars, Not Parameters: LilMoo Compact Language Model for Hindi
Shiza Fatimah, Aniket Sen, Sophia Falk +3
The dominance of large multilingual foundation models has widened linguistic inequalities in Natural Language Processing (NLP), often leaving low-resource languages underrepresente…
Understanding Artificial Theory of Mind: Perturbed Tasks and Reasoning in Large Language Models
Christian Nickel, Laura Schrewe, Florian Mai +1
Theory of Mind (ToM) refers to an agent's ability to model the internal states of others. Contributing to the debate whether large language models (LLMs) exhibit genuine ToM capabi…
IKnow: Instruction-Knowledge-Aware Continual Pretraining for Effective Domain Adaptation
Tianyi Zhang, Florian Mai, Lucie Flek
Continual pretraining promises to adapt large language models (LLMs) to new domains using only unlabeled test-time data, but naively applying standard self-supervised objectives to…
In-Training Defenses against Emergent Misalignment in Language Models
David Kaczér, Magnus Jørgenvåg, Clemens Vetter +4
Fine-tuning lets practitioners repurpose aligned large language models (LLMs) for new domains, yet recent work reveals emergent misalignment (EM): Even a small, domain-specific fin…
Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models
Mehdi Ali, Manuel Brack, Max Lübbering +15
High-quality multilingual training data is essential for effectively pretraining large language models (LLMs). Yet, the availability of suitable open-source multilingual datasets r…