3 papers
cs.CL2025
Apertus: Democratizing Open and Compliant LLMs for Global Language Environments
Project Apertus, Alejandro Hernández-Cano, Alexander Hägele +100
We present Apertus, a fully open suite of large language models (LLMs) designed to address two systemic shortcomings in today's open model ecosystem: data compliance and multilingu…
cs.LG2025
Deriving Activation Functions Using Integration
Allen Hao Huang, Imanol Schlag
Our work proposes a novel approach to designing activation functions by focusing on their gradients and deriving the corresponding activation functions using integration. We introd…
cs.NE2024
Expanded Gating Ranges Improve Activation Functions
Allen Hao Huang
Activation functions are core components of all deep learning architectures. Currently, the most popular activation functions are smooth ReLU variants like GELU and SiLU. These are…