5 papers
Kolibri: A Sovereign European Model on the Pareto Frontier
Aleph Alpha, :, Moritz Althaus +65
This report introduces Kolibri, an English-German mixture-of-experts transformer with 78.1B total parameters and 3.46B active parameters per token, released as open weights under t…
Beyond Prompt and Parser Engineering: Soft-Prompt Tuning for Automated LLM Evaluation
Selen Erkan, Bastian Boll, Kristian Kersting +2
Benchmark scores often misrepresent a large language model's (LLM's) knowledge, because they rely, e.g., on the model's ability to follow specific formatting requirements. This esp…
AtManRL: Towards Faithful Reasoning via Differentiable Attention Saliency
Max Henning Höth, Kristian Kersting, Björn Deiseroth +1
Large language models (LLMs) increasingly rely on chain-of-thought (CoT) reasoning to solve complex tasks, yet a reasoning trace may merely accompany the final answer rather than c…
A Family of LLMs Liberated from Static Vocabularies
Aleph Alpha, :, Adnen Abdessaied +35
Tokenization is a central component of natural language processing in current large language models (LLMs), enabling models to convert raw text into processable units. Although lea…
Bounding Hallucinations: Merlin-Arthur Protocols for Mutual-Information Bounds in Language Models
Björn Deiseroth, Max Henning Höth, Kristian Kersting +1
Retrieval-augmented generation (RAG) relies on retrieved context to guide large language models (LLM), yet treats the retrieval as a heuristic rather than verifiable evidence -- le…