95 citations · 243 across the 10 of their papers we have counts for
3 papers · 1 filter
A Sovereign, Open-Source Foundation Model for German and English
Soofi-Team, :, Benedikt Droste +30
We present Soofi S 30B-A3B, a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba Transformer foundation model for German and English. Its hybrid design activates only 3B…
The Loss Is Not Enough: Sampling Conditions and Inductive Bias in Contrastive Representation Learning
Justinas Zaliaduonis, Patrick Putzky, Till Richter +1
Contrastive learning has become a leading paradigm for self-supervised representation learning, yet the conditions under which it recovers meaningful latent geometry remain incompl…
Float8@2bits: Entropy Coding Enables Data-Free Model Compression
Patrick Putzky, Martin Genzel, Mattes Mollenhauer +3
Post-training compression is currently divided into two contrasting regimes. On the one hand, fast, data-free, and model-agnostic methods (e.g., NF4 or HQQ) offer maximum accessibi…