5 citations · 10 across the 18 of their papers we have counts for
18 papers
Beyond the Best Guess: Improving LLM Solution Coverage with Evolution Strategies
Conor F. Hayes, Elliot Meyerson, Kajetan Schweighofer +4
Large Language Models (LLMs) are increasingly deployed in discovery domains such as math and science. The usual approach is to present the problem to the model and use its answer a…
Automated Background Swapping for Robustness against Spurious Backgrounds
Cesar Roder, Kajetan Schweighofer
Classifiers based on Deep Neural Networks exhibit strong performance across domains, yet can fail catastrophically if they rely on spurious correlations, i.e., features that are pr…
RREDCoT: Segment-Level Reward Redistribution for Reasoning Models
Mykyta Ielanskyi, Kajetan Schweighofer, Lukas Aichberger +1
Recent advancements in reasoning language models have been driven by Reinforcement Learning (RL) fine-tuning. Most often, these rely on the Group Relative Policy Optimization (GRPO…
Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies
Kajetan Schweighofer, Conor F. Hayes, Roberto Dailey +2
Evolution Strategies (ES) has recently emerged as a competitive alternative to reinforcement learning (RL) for large language model (LLM) fine-tuning, offering advantages through s…
Efficient Pre-Training of LLMs through Truncated SVD Representations
Kaivan Kamali, Kajetan Schweighofer, Hormoz Shahrzad +3
LLM pretraining is extremely costly; therefore, parameter-efficient LLM architectures have recently emerged as a compelling research direction. One such promising approach is to re…
Position: agentic AI orchestration should be Bayes-consistent
Theodore Papamarkou, Pierre Alquier, Matthias Bauer +27
LLMs excel at predictive tasks and complex reasoning tasks, but many high-value deployments rely on decisions under uncertainty, for example, which tool to call, which expert to co…
Addressing Pitfalls in the Evaluation of Uncertainty Estimation Methods for Natural Language Generation
Mykyta Ielanskyi, Kajetan Schweighofer, Lukas Aichberger +1
Hallucinations are a common issue that undermine the reliability of large language models (LLMs). Recent studies have identified a specific subset of hallucinations, known as confa…
xLSTM Scaling Laws: Competitive Performance with Linear Time-Complexity
Maximilian Beck, Kajetan Schweighofer, Sebastian Böck +2
Scaling laws play a central role in the success of Large Language Models (LLMs), enabling the prediction of model performance relative to compute budgets prior to training. While T…
Uncertainty Quantification for Regression using Proper Scoring Rules
Alexander Fishkov, Kajetan Schweighofer, Mykyta Ielanskyi +3
Quantifying uncertainty of machine learning model predictions is essential for reliable decision-making, especially in safety-critical applications. Recently, uncertainty quantific…
Safe and Certifiable AI Systems: Concepts, Challenges, and Lessons Learned
Kajetan Schweighofer, Barbara Brune, Lukas Gruber +11
There is an increasing adoption of artificial intelligence in safety-critical applications, yet practical schemes for certifying that AI systems are safe, lawful and socially accep…
ImageSet2Text: Describing Sets of Images through Text
Piera Riccio, Francesco Galati, Kajetan Schweighofer +2
In the era of large-scale visual data, understanding collections of images is a challenging yet important task. To this end, we introduce ImageSet2Text, a novel method to automatic…
Rethinking Uncertainty Estimation in LLMs: A Principled Single-Sequence Measure
Lukas Aichberger, Kajetan Schweighofer, Sepp Hochreiter
Large Language Models (LLMs) are increasingly employed in real-world applications, driving the need to evaluate the trustworthiness of their generated text. To this end, reliable u…
The Disparate Benefits of Deep Ensembles
Kajetan Schweighofer, Adrian Arnaiz-Rodriguez, Sepp Hochreiter +1
Ensembles of Deep Neural Networks, Deep Ensembles, are widely used as a simple way to boost predictive performance. However, their impact on algorithmic fairness is not well unders…
On Information-Theoretic Measures of Predictive Uncertainty
Kajetan Schweighofer, Lukas Aichberger, Mykyta Ielanskyi +1
Reliable estimation of predictive uncertainty is crucial for machine learning applications, particularly in high-stakes scenarios where hedging against risks is essential. Despite…
Improving Uncertainty Estimation through Semantically Diverse Language Generation
Lukas Aichberger, Kajetan Schweighofer, Mykyta Ielanskyi +1
Large language models (LLMs) can suffer from hallucinations when generating text. These hallucinations impede various applications in society and industry by making LLMs untrustwor…
Introducing an Improved Information-Theoretic Measure of Predictive Uncertainty
Kajetan Schweighofer, Lukas Aichberger, Mykyta Ielanskyi +1
Applying a machine learning model for decision-making in the real world requires to distinguish what the model knows from what it does not. A critical factor in assessing the knowl…
Quantification of Uncertainty with Adversarial Models
Kajetan Schweighofer, Lukas Aichberger, Mykyta Ielanskyi +2
Quantifying uncertainty is important for actionable predictions in real-world applications. A crucial part of predictive uncertainty quantification is the estimation of epistemic u…
A Dataset Perspective on Offline Reinforcement Learning
Kajetan Schweighofer, Andreas Radler, Marius-Constantin Dinu +5
The application of Reinforcement Learning (RL) in real world environments can be expensive or risky due to sub-optimal policies during training. In Offline RL, this problem is avoi…