4 papers
IKnow: Instruction-Knowledge-Aware Continual Pretraining for Effective Domain Adaptation
Tianyi Zhang, Florian Mai, Lucie Flek
Continual pretraining promises to adapt large language models (LLMs) to new domains using only unlabeled test-time data, but naively applying standard self-supervised objectives to…
Survey-to-Behavior: Downstream Alignment of Human Values in LLMs via Survey Questions
Shangrui Nie, Florian Mai, David Kaczér +3
Large language models implicitly encode preferences over human values, yet steering them often requires large training data. In this work, we investigate a simple approach: Can we…
Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models
Mehdi Ali, Manuel Brack, Max Lübbering +15
High-quality multilingual training data is essential for effectively pretraining large language models (LLMs). Yet, the availability of suitable open-source multilingual datasets r…
Superalignment with Dynamic Human Values
Florian Mai, David Kaczér, Nicholas Kluge Corrêa +1
Two core challenges of alignment are 1) scalable oversight and 2) accounting for the dynamic nature of human values. While solutions like recursive reward modeling address 1), they…