1 citations · 1 across the 13 of their papers we have counts for
6 papers · 1 filter
Fetch Me If You Can: A Comprehensive Analysis of Fetch Metadata
Pedro Bernardo, Alex Ploner, Marco Squarcina +2
Effective server-side defenses against cross-site attacks depend on understanding the context in which browser requests are generated. However, servers often lack the information n…
Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors
Alexander Hägele, Alejandro Hernández-Cano, Atli Kosson +1
Modern neural network training relies on optimizers such as Adam and Muon which act on each weight matrix as a single object. Yet every weight matrix carries two distinct quantitie…
Tying the Loop -- Tied Expert Layers in Mixture-of-Experts Language Models
Martin Jaggi
Mixture-of-Experts (MoE) architectures efficiently scale Large Language Models (LLMs) by activating only a small fraction of their experts per token, yet the full parameter count -…
Apertus LLM Family Expansion via Distillation and Quantization
Andrei Panferov, Davit Melikidze, Martin Jaggi +1
The wide adoption of LLMs has led to their use in great variety of applications and scenarios, such as chatbot assistants and data annotation, creating the need for the models to s…
Toward Cross-Lingual Quality Classifiers for Multilingual Pretraining Data Selection
Yassine Turki, Vinko Sabolčec, Bettina Messmer +1
As Large Language Models (LLMs) scale, data curation has shifted from maximizing volume to optimizing the signal-to-noise ratio by performing quality filtering. However, for many l…
An Engineering Journey Training Large Language Models at Scale on Alps: The Apertus Experience
Jonathan Coles, Stefano Schuppli, Lukas Drescher +20
Large Language Models (LLMs) have surged as a transformative technology for science and society, prompting governments worldwide to pursue sovereign AI capabilities that ensure dat…