4 papers
A Sovereign, Open-Source Foundation Model for German and English
The Soofi-Team, Soofi-Team, : +31
We present Soofi S 30B-A3B, a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba Transformer foundation model for German and English. Its hybrid design activates only 3B…
New Encoders for German Trained from Scratch: Comparing ModernGBERT with Converted LLM2Vec Models
Julia Wunderle, Anton Ehrmanntraut, Jan Pfister +2
Encoders remain essential for efficient German NLP and NLU scenarios despite the rise of decoder-only LLMs. This work studies two routes to high-quality German encoders under ident…
LLäMmlein: Transparent, Compact and Competitive German-Only Language Models from Scratch
Jan Pfister, Julia Wunderle, Andreas Hotho
We create two German-only decoder models, LLäMmlein 120M and 1B, transparently from scratch and publish them, along with the training data, for the German NLP research community t…
Modeling and Analyzing the Influence of Non-Item Pages on Sequential Next-Item Prediction
Elisabeth Fischer, Albin Zehe, Andreas Hotho +1
Analyzing sequences of interactions between users and items, sequential recommendation models can learn user intent and make predictions about the next item. Next to item interacti…