134 citations · 776 across the 124 of their papers we have counts for
24 papers · 1 filter
AutoBaxBuilder: Bootstrapping Code Security Benchmarking
Tobias von Arx, Niels Mündler, Mark Vero +2
As large language models (LLMs) see wide adoption in software engineering, the reliable assessment of the correctness and security of LLM-generated code is crucial. Notably, prior…
Dual Randomized Smoothing: Beyond Global Noise Variance
Chenhao Sun, Yuhao Mao, Martin Vechev
Randomized Smoothing (RS) is a prominent technique for certifying the robustness of neural networks against adversarial perturbations. With RS, achieving high accuracy at small rad…
SPEAR++: Scaling Gradient Inversion via Sparsely-Used Dictionary Learning
Alexander Bakarsky, Dimitar I. Dimitrov, Maximilian Baader +1
Federated Learning has seen an increased deployment in real-world scenarios recently, as it enables the distributed training of machine learning models without explicit data sharin…
Pay Attention to the Triggers: Constructing Backdoors That Survive Distillation
Giovanni De Muri, Mark Vero, Robin Staab +1
LLMs are often used by downstream users as teacher models for knowledge distillation, compressing their capabilities into memory-efficient models. However, as these teacher models…
BrokenMath: A Benchmark for Sycophancy in Theorem Proving with LLMs
Ivo Petrov, Jasper Dekoninck, Martin Vechev
Large language models (LLMs) have recently shown strong performance on mathematical benchmarks. At the same time, they are prone to hallucination and sycophancy, often providing co…
Adaptive Generation of Bias-Eliciting Questions for LLMs
Robin Staab, Jasper Dekoninck, Maximilian Baader +1
Large language models (LLMs) are now widely deployed in user-facing applications, reaching hundreds of millions of users worldwide. Despite their widespread adoption, growing relia…
Fewer Weights, More Problems: A Practical Attack on LLM Pruning
Kazuki Egashira, Robin Staab, Thibaud Gloaguen +2
Model pruning, i.e., removing a subset of model weights, has become a prominent approach to reducing the memory footprint of large language models (LLMs) during inference. Notably,…
Watermarking Diffusion Language Models
Thibaud Gloaguen, Robin Staab, Nikola Jovanović +1
We introduce the first watermark tailored for diffusion language models (DLMs), an emergent LLM paradigm able to generate tokens in arbitrary order, in contrast to standard autoreg…
Constrained Decoding of Diffusion LLMs with Context-Free Grammars
Niels Mündler, Jasper Dekoninck, Martin Vechev
Large language models (LLMs) have shown promising performance across diverse domains. Many practical applications of LLMs, such as code completion and structured data extraction, r…
SoK: Data Minimization in Machine Learning
Robin Staab, Nikola Jovanović, Kimberly Mai +4
Data minimization (DM) describes the principle of collecting only the data strictly necessary for a given task. It is a foundational principle across major data protection regulati…
The Open Proof Corpus: A Large-Scale Study of LLM-Generated Mathematical Proofs
Jasper Dekoninck, Ivo Petrov, Kristian Minchev +13
In recent months, large language models (LLMs) have made significant progress in mathematical proof generation, but further advancement is hindered by the lack of a large-scale, hi…
Watermarking Autoregressive Image Generation
Nikola Jovanović, Ismail Labiad, Tomáš Souček +2
Watermarking the outputs of generative models has emerged as a promising approach for tracking their provenance. Despite significant interest in autoregressive image generation mod…
Type-Constrained Code Generation with Language Models
Niels Mündler, Jingxuan He, Hao Wang +3
Large language models (LLMs) have achieved notable success in code generation. However, they still frequently produce uncompilable output because their next-token inference procedu…
MathArena: Evaluating LLMs on Uncontaminated Math Competitions
Mislav Balunović, Jasper Dekoninck, Ivo Petrov +2
The rapid advancement of reasoning capabilities in large language models (LLMs) has led to notable improvements on mathematical benchmarks. However, many of the most commonly used…
MixAT: Combining Continuous and Discrete Adversarial Training for LLMs
Csaba Dékány, Stefan Balauca, Robin Staab +2
Despite recent efforts in Large Language Model (LLM) safety and alignment, current adversarial attacks on frontier LLMs can still consistently force harmful generations. Although a…
LLM Fingerprinting via Semantically Conditioned Watermarks
Thibaud Gloaguen, Robin Staab, Nikola Jovanović +1
Most LLM fingerprinting methods teach the model to respond to a few fixed queries with predefined atypical responses (keys). This memorization often does not survive common deploym…
Watch your steps: Dormant Adversarial Behaviors that Activate upon LLM Finetuning
Thibaud Gloaguen, Mark Vero, Robin Staab +1
Finetuning open-weight Large Language Models (LLMs) is standard practice for achieving task-specific performance improvements. Until now, finetuning has been regarded as a controll…
ToolFuzz -- Automated Agent Tool Testing
Ivan Milev, Mislav Balunović, Maximilian Baader +1
Large Language Model (LLM) Agents leverage the advanced reasoning capabilities of LLMs in real-world applications. To interface with an environment, these agents often rely on tool…
Automated Benchmark Generation for Repository-Level Coding Tasks
Konstantinos Vergopoulos, Mark Niklas Müller, Martin Vechev
Code Agent development is an extremely active research area, where a reliable performance metric is critical for tracking progress and guiding new developments. This demand is unde…
GRAIN: Exact Graph Reconstruction from Gradients
Maria Drencheva, Ivo Petrov, Maximilian Baader +2
Federated learning claims to enable collaborative model training among multiple clients with data privacy by transmitting gradient updates instead of the actual client data. Howeve…
Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad
Ivo Petrov, Jasper Dekoninck, Lyuben Baltadzhiev +5
Recent math benchmarks for large language models (LLMs) such as MathArena indicate that state-of-the-art reasoning models achieve impressive performance on mathematical competition…
Towards Watermarking of Open-Source LLMs
Thibaud Gloaguen, Nikola Jovanović, Robin Staab +1
While watermarks for closed LLMs have matured and have been included in large-scale deployments, these methods are not applicable to open-source models, which allow users full cont…
BaxBench: Can LLMs Generate Correct and Secure Backends?
Mark Vero, Niels Mündler, Victor Chibotaru +5
Automatic program generation has long been a fundamental challenge in computer science. Recent benchmarks have shown that large language models (LLMs) can effectively generate code…
MathConstruct: Challenging LLM Reasoning with Constructive Proofs
Mislav Balunović, Jasper Dekoninck, Nikola Jovanović +2
While Large Language Models (LLMs) demonstrate impressive performance in mathematics, existing math benchmarks come with significant limitations. Many focus on problems with fixed…