1 citations · 1 across the 7 of their papers we have counts for
8 papers
Kolibri: A Sovereign European Model on the Pareto Frontier
Aleph Alpha, :, Moritz Althaus +65
This report introduces Kolibri, an English-German mixture-of-experts transformer with 78.1B total parameters and 3.46B active parameters per token, released as open weights under t…
A Family of LLMs Liberated from Static Vocabularies
Aleph Alpha, :, Adnen Abdessaied +35
Tokenization is a central component of natural language processing in current large language models (LLMs), enabling models to convert raw text into processable units. Although lea…
When Guessing is Rewarded: Rethinking Language Model Evaluation with Distributional Uncertainty Scoring
Thomas F Burns
Standard language model evaluation assigns scores to single predicted answers, rewarding high-confidence responses regardless of how residual probability mass is distributed over a…
Aleph-Alpha-GermanWeb: Improving German-language LLM pre-training with model-based data curation and synthetic data generation
Thomas F Burns, Letitia Parcalabescu, Stephan Wäldchen +5
Scaling data quantity is essential for large language models (LLMs), yet recent findings show that data quality can significantly boost performance and training efficiency. We intr…
Associative memory inspires improvements for in-context learning using a novel attention residual stream architecture
Thomas F Burns, Tomoki Fukai, Christopher J Earls
Large language models (LLMs) demonstrate an impressive ability to utilise information within the context of their input sequences to appropriately respond to data unseen by the LLM…
Generative AI Policy and Governance Considerations for Health Security in Southeast Asia
Thomas F Burns
Southeast Asia is a geopolitically and socio-economically significant region with unique challenges and opportunities. Intensifying progress in generative AI against a backdrop of…
Semantically-correlated memories in a dense associative model
Thomas F Burns
I introduce a novel associative memory model named Correlated Dense Associative Memory (CDAM), which integrates both auto- and hetero-association in a unified framework for continu…
Altered Topological Structure of the Brain White Matter in Maltreated Children through Topological Data Analysis
Moo K. Chung, Tahmineh Azizi, Jamie L. Hanson +3
Childhood maltreatment may adversely affect brain development and consequently influence behavioral, emotional, and psychological patterns during adulthood. In this study, we propo…