74 citations · 132 across the 14 of their papers we have counts for
16 papers
TestSuiteBench: Coding Agents Struggle to Generate Strong Test Suites from Specifications
Kazuki Egashira, Niels Mündler-Sasahara, Mark Niklas Müller +1
Coding agents are increasingly used to create large software projects. While code-generation benchmarks have scaled accordingly to full applications, test-generation benchmarks rem…
Coding Agents Don't Know When to Act
Thibaud Gloaguen, Niels Mündler, Mark Müller +2
Coding agents are increasingly deployed to autonomously maintain software, including to resolve user-reported issues: a bug report comes in and the agent creates a patch to address…
Automated Benchmark Generation for Repository-Level Coding Tasks
Konstantinos Vergopoulos, Mark Niklas Müller, Martin Vechev
Code Agent development is an extremely active research area, where a reliable performance metric is critical for tracking progress and guiding new developments. This demand is unde…
Diagnosing Robotics Systems Issues with Large Language Models
Jordis Emilia Herrmann, Aswath Mandakath Gopinath, Mikael Norrlof +1
Quickly resolving issues reported in industrial applications is crucial to minimize economic impact. However, the required data analysis makes diagnosing the underlying root causes…
Average Certified Radius is a Poor Metric for Randomized Smoothing
Chenhao Sun, Yuhao Mao, Mark Niklas Müller +1
Randomized smoothing (RS) is popular for providing certified robustness guarantees against adversarial attacks. The average certified radius (ACR) has emerged as a widely used metr…
Mitigating Catastrophic Forgetting in Language Transfer via Model Merging
Anton Alexandrov, Veselin Raychev, Mark Niklas Müller +3
As open-weight large language models (LLMs) achieve ever more impressive performances across a wide range of tasks in English, practitioners aim to adapt these models to different…
Certification for Differentially Private Prediction in Gradient-Based Training
Matthew Wicker, Philip Sosnin, Igor Shilov +5
We study private prediction where differential privacy is achieved by adding noise to the outputs of a non-private model. Existing methods rely on noise proportional to the global…
SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents
Niels Mündler, Mark Niklas Müller, Jingxuan He +1
Rigorous software testing is crucial for developing and maintaining high-quality code, making automated test generation a promising avenue for both improving software quality and b…
Certified Robustness to Data Poisoning in Gradient-Based Training
Philip Sosnin, Mark N. Müller, Maximilian Baader +2
Modern machine learning pipelines leverage large amounts of public data, making it infeasible to guarantee data quality and leaving models open to poisoning and backdoor attacks. P…
ConStat: Performance-Based Contamination Detection in Large Language Models
Jasper Dekoninck, Mark Niklas Müller, Martin Vechev
Public benchmarks play an essential role in the evaluation of large language models. However, data contamination can lead to inflated performance, rendering them unreliable for mod…
DAGER: Exact Gradient Inversion for Large Language Models
Ivo Petrov, Dimitar I. Dimitrov, Maximilian Baader +2
Federated learning works by aggregating locally computed gradients from multiple clients, thus enabling collaborative training without sharing private client data. However, prior w…
Expressivity of ReLU-Networks under Convex Relaxations
Maximilian Baader, Mark Niklas Müller, Yuhao Mao +1
Convex relaxations are a key component of training and certifying provably safe neural networks. However, despite substantial progress, a wide and poorly understood accuracy gap to…
The Third International Verification of Neural Networks Competition (VNN-COMP 2022): Summary and Results
Mark Niklas Müller, Christopher Brix, Stanley Bak +2
This report summarizes the 3rd International Verification of Neural Networks Competition (VNN-COMP 2022), held as a part of the 5th Workshop on Formal Methods for ML-Enabled Autono…
Complete Verification via Multi-Neuron Relaxation Guided Branch-and-Bound
Claudio Ferrari, Mark Niklas Muller, Nikola Jovanovic +1
State-of-the-art neural network verifiers are fundamentally based on one of two paradigms: either encoding the whole verification problem via tight multi-neuron convex relaxations…
Boosting Randomized Smoothing with Variance Reduced Classifiers
Miklós Z. Horváth, Mark Niklas Müller, Marc Fischer +1
Randomized Smoothing (RS) is a promising method for obtaining robustness certificates by evaluating a base model under noise. In this work, we: (i) theoretically motivate why ensem…
PRIMA: General and Precise Neural Network Certification via Scalable Convex Hull Approximations
Mark Niklas Müller, Gleb Makarchuk, Gagandeep Singh +2
Formal verification of neural networks is critical for their safe adoption in real-world applications. However, designing a precise and scalable verifier which can handle different…