5 papers
Optimistic Dual Averaging Unifies Modern Optimizers
Thomas Pethick, Wanyun Xie, Roman Machacek +1
We introduce SODA, a generalization of Optimistic Dual Averaging, which provides a common perspective on state-of-the-art optimizers like Muon, Lion, AdEMAMix and NAdam, showing th…
On the Role of Batch Size in Stochastic Conditional Gradient Methods
Rustem Islamov, Roman Machacek, Aurelien Lucchi +3
We study the role of batch size in stochastic conditional gradient methods under a -Kurdyka-Åojasiewicz (-KL) condition. Focusing on momentum-based stochastic conditional…
Apertus: Democratizing Open and Compliant LLMs for Global Language Environments
Project Apertus, Alejandro Hernández-Cano, Alexander Hägele +100
We present Apertus, a fully open suite of large language models (LLMs) designed to address two systemic shortcomings in today's open model ecosystem: data compliance and multilingu…
The Impact of Fine-tuning Large Language Models on Automated Program Repair
Roman MacháÄek, Anastasiia Grishina, Max Hort +1
Automated Program Repair (APR) uses various tools and techniques to help developers achieve functional and error-free code faster. In recent years, Large Language Models (LLMs) hav…
Explaining GitHub Actions Failures with Large Language Models: Challenges, Insights, and Limitations
Pablo Valenzuela-Toledo, Chuyue Wu, Sandro Hernandez +4
GitHub Actions (GA) has become the de facto tool that developers use to automate software workflows, seamlessly building, testing, and deploying code. Yet when GA fails, it disrupt…