1 citations · 1 across the 3 of their papers we have counts for
4 papers
Kolibri: A Sovereign European Model on the Pareto Frontier
Aleph Alpha, :, Moritz Althaus +65
This report introduces Kolibri, an English-German mixture-of-experts transformer with 78.1B total parameters and 3.46B active parameters per token, released as open weights under t…
Good Pretraining, Bad SFT: Checkpoint Quality Across the Training Stack
Sohir Maskey, Philipp Scholl, Jonas Knupp +2
Language-model checkpoints are commonly selected by pretraining loss or benchmark scores, assuming that the highest-scoring checkpoint will remain the best starting point for subse…
Depth-Recurrent Attention Mixtures: Giving Latent Reasoning the Attention it Deserves
Jonas Knupp, Jan Hendrik Metzen, Jeremias Bohn +2
Depth-recurrence facilitates latent reasoning by sharing parameters across depths. However, prior work lacks combined FLOP-, parameter-, and memory-matched baselines, underutilizes…
RadEx: A Framework for Structured Information Extraction from Radiology Reports based on Large Language Models
Daniel Reichenpfader, Jonas Knupp, André Sander +1
Annually and globally, over three billion radiography examinations and computer tomography scans result in mostly unstructured radiology reports containing free text. Despite the p…