278 citations · 846 across the 26 of their papers we have counts for
22 papers · 1 filter
Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols
Mikhail Terekhov, Alexander Panfilov, Daniil Dzenhaliou +4
AI control protocols serve as a defense mechanism to stop untrusted LLM agents from causing harm in autonomous settings. Prior work treats this as a security problem, stress testin…
Strategic Dishonesty Can Undermine AI Safety Evaluations of Frontier LLMs
Alexander Panfilov, Evgenii Kortukov, Kristina Nikolić +6
Large language model (LLM) developers aim for their models to be honest, helpful, and harmless. However, when faced with malicious requests, models are trained to refuse, sacrifici…
Exploring Memorization and Copyright Violation in Frontier LLMs: A Study of the New York Times v. OpenAI 2023 Lawsuit
Joshua Freeman, Chloe Rippe, Edoardo Debenedetti +1
Copyright infringement in frontier LLMs has received much attention recently due to the New York Times v. OpenAI lawsuit, filed in December 2023. The New York Times claims that GPT…
Scaling Compute Is Not All You Need for Adversarial Robustness
Edoardo Debenedetti, Zishen Wan, Maksym Andriushchenko +3
The last six years have witnessed significant progress in adversarially robust deep learning. As evidenced by the CIFAR-10 dataset category in RobustBench benchmark, the accuracy u…
Critical Influence of Overparameterization on Sharpness-aware Minimization
Sungbin Shin, Dongyeop Lee, Maksym Andriushchenko +1
Sharpness-Aware Minimization (SAM) has attracted considerable attention for its effectiveness in improving generalization in deep neural network training by explicitly minimizing s…
Layer-wise Linear Mode Connectivity
Linara Adilova, Maksym Andriushchenko, Michael Kamp +2
Averaging neural network parameters is an intuitive method for fusing the knowledge of two independent models. It is most prominently used in federated learning. If models are aver…
Transferable Adversarial Robustness for Categorical Data via Universal Robust Embeddings
Klim Kireev, Maksym Andriushchenko, Carmela Troncoso +1
Research on adversarial robustness is primarily focused on image and text data. Yet, many scenarios in which lack of robustness can result in serious risks, such as fraud detection…
Sharpness-Aware Minimization Leads to Low-Rank Features
Maksym Andriushchenko, Dara Bahri, Hossein Mobahi +1
Sharpness-aware minimization (SAM) is a recently proposed method that minimizes the sharpness of the training loss of a neural network. While its generalization improvement is well…
A Modern Look at the Relationship between Sharpness and Generalization
Maksym Andriushchenko, Francesco Croce, Maximilian Müller +2
Sharpness of minima is a promising quantity that can correlate with generalization in deep networks and, when optimized during training, can improve generalization. However, standa…
SGD with Large Step Sizes Learns Sparse Features
Maksym Andriushchenko, Aditya Varre, Loucas Pillaud-Vivien +1
We showcase important features of the dynamics of the Stochastic Gradient Descent (SGD) in the training of neural networks. We present empirical observations that commonly used lar…
Towards Understanding Sharpness-Aware Minimization
Maksym Andriushchenko, Nicolas Flammarion
Sharpness-Aware Minimization (SAM) is a recent training method that relies on worst-case weight perturbations which significantly improves generalization in various settings. We ar…
On the effectiveness of adversarial training against common corruptions
Klim Kireev, Maksym Andriushchenko, Nicolas Flammarion
The literature on robustness towards common corruptions shows no consensus on whether adversarial training can improve the performance in this setting. First, we show that, when us…
RobustBench: a standardized adversarial robustness benchmark
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag +5
As a research community, we are still lacking a systematic understanding of the progress on adversarial robustness which often makes it hard to identify the most promising ideas in…
Understanding and Improving Fast Adversarial Training
Maksym Andriushchenko, Nicolas Flammarion
A recent line of work focused on making adversarial training computationally efficient for deep learning models. In particular, Wong et al. (2020) showed that -adversa…
Sparse-RS: a versatile framework for query-efficient sparse black-box adversarial attacks
Francesco Croce, Maksym Andriushchenko, Naman D. Singh +2
We propose a versatile framework based on random search, Sparse-RS, for score-based sparse targeted and untargeted attacks in the black-box setting. Sparse-RS does not rely on subs…
On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines
Marius Mosbach, Maksym Andriushchenko, Dietrich Klakow
Fine-tuning pre-trained transformer-based language models such as BERT has become a common practice dominating leaderboards across various NLP benchmarks. Despite the strong empiri…
Square Attack: a query-efficient black-box adversarial attack via random search
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion +1
We propose the Square Attack, a score-based black-box - and -adversarial attack that does not rely on local gradient information and thus is not affected by gradient…
Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks
Maksym Andriushchenko, Matthias Hein
The problem of adversarial robustness has been studied extensively for neural networks. However, for boosted decision trees and decision stumps there are almost no results, even th…
Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem
Matthias Hein, Maksym Andriushchenko, Julian Bitterwolf
Classifiers used in the wild, in particular for safety-critical systems, should not only have good generalization properties but also should know when they don't know, in particula…
Logit Pairing Methods Can Fool Gradient-Based Attacks
Marius Mosbach, Maksym Andriushchenko, Thomas Trost +2
Recently, Kannan et al. [2018] proposed several logit regularization methods to improve the adversarial robustness of classifiers. We show that the computationally fast methods the…
Provable Robustness of ReLU networks via Maximization of Linear Regions
Francesco Croce, Maksym Andriushchenko, Matthias Hein
It has been shown that neural network classifiers are not robust. This raises concerns about their usage in safety-critical systems. We propose in this paper a regularization schem…
Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation
Matthias Hein, Maksym Andriushchenko
Recent work has shown that state-of-the-art classifiers are quite brittle, in the sense that a small adversarial change of an originally with high confidence correctly classified i…