278 citations · 846 across the 26 of their papers we have counts for
26 papers
Stealing Reasoning Traces from Proprietary LLM APIs
Alexander Panfilov, David Schmotz, Ilia Shumailov +5
Leading large language model providers now conceal their models' step-by-step reasoning, or chain-of-thought, to protect intellectual property and limit information leakage. Rather…
Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols
Mikhail Terekhov, Alexander Panfilov, Daniil Dzenhaliou +4
AI control protocols serve as a defense mechanism to stop untrusted LLM agents from causing harm in autonomous settings. Prior work treats this as a security problem, stress testin…
Strategic Dishonesty Can Undermine AI Safety Evaluations of Frontier LLMs
Alexander Panfilov, Evgenii Kortukov, Kristina Nikolić +6
Large language model (LLM) developers aim for their models to be honest, helpful, and harmless. However, when faced with malicious requests, models are trained to refuse, sacrifici…
Capability-Based Scaling Trends for LLM-Based Red-Teaming
Alexander Panfilov, Paul Kassianik, Maksym Andriushchenko +1
As large language models grow in capability and agency, identifying vulnerabilities through red-teaming becomes vital for safe deployment. However, traditional prompt-engineering a…
Exploring Memorization and Copyright Violation in Frontier LLMs: A Study of the New York Times v. OpenAI 2023 Lawsuit
Joshua Freeman, Chloe Rippe, Edoardo Debenedetti +1
Copyright infringement in frontier LLMs has received much attention recently due to the New York Times v. OpenAI lawsuit, filed in December 2023. The New York Times claims that GPT…
Beyond Performance: Quantifying and Mitigating Label Bias in LLMs
Yuval Reif, Roy Schwartz
Large language models (LLMs) have shown remarkable adaptability to diverse tasks, by leveraging context prompts containing instructions, or minimal input-output examples. However,…
Scaling Compute Is Not All You Need for Adversarial Robustness
Edoardo Debenedetti, Zishen Wan, Maksym Andriushchenko +3
The last six years have witnessed significant progress in adversarially robust deep learning. As evidenced by the CIFAR-10 dataset category in RobustBench benchmark, the accuracy u…
Critical Influence of Overparameterization on Sharpness-aware Minimization
Sungbin Shin, Dongyeop Lee, Maksym Andriushchenko +1
Sharpness-Aware Minimization (SAM) has attracted considerable attention for its effectiveness in improving generalization in deep neural network training by explicitly minimizing s…
Layer-wise Linear Mode Connectivity
Linara Adilova, Maksym Andriushchenko, Michael Kamp +2
Averaging neural network parameters is an intuitive method for fusing the knowledge of two independent models. It is most prominently used in federated learning. If models are aver…
Transferable Adversarial Robustness for Categorical Data via Universal Robust Embeddings
Klim Kireev, Maksym Andriushchenko, Carmela Troncoso +1
Research on adversarial robustness is primarily focused on image and text data. Yet, many scenarios in which lack of robustness can result in serious risks, such as fraud detection…
Sharpness-Aware Minimization Leads to Low-Rank Features
Maksym Andriushchenko, Dara Bahri, Hossein Mobahi +1
Sharpness-aware minimization (SAM) is a recently proposed method that minimizes the sharpness of the training loss of a neural network. While its generalization improvement is well…
A Modern Look at the Relationship between Sharpness and Generalization
Maksym Andriushchenko, Francesco Croce, Maximilian Müller +2
Sharpness of minima is a promising quantity that can correlate with generalization in deep networks and, when optimized during training, can improve generalization. However, standa…
SGD with Large Step Sizes Learns Sparse Features
Maksym Andriushchenko, Aditya Varre, Loucas Pillaud-Vivien +1
We showcase important features of the dynamics of the Stochastic Gradient Descent (SGD) in the training of neural networks. We present empirical observations that commonly used lar…
Towards Understanding Sharpness-Aware Minimization
Maksym Andriushchenko, Nicolas Flammarion
Sharpness-Aware Minimization (SAM) is a recent training method that relies on worst-case weight perturbations which significantly improves generalization in various settings. We ar…
ARIA: Adversarially Robust Image Attribution for Content Provenance
Maksym Andriushchenko, Xiaoyang Rebecca Li, Geoffrey Oxholm +4
Image attribution -- matching an image back to a trusted source -- is an emerging tool in the fight against online misinformation. Deep visual fingerprinting models have recently b…
On the effectiveness of adversarial training against common corruptions
Klim Kireev, Maksym Andriushchenko, Nicolas Flammarion
The literature on robustness towards common corruptions shows no consensus on whether adversarial training can improve the performance in this setting. First, we show that, when us…
RobustBench: a standardized adversarial robustness benchmark
Francesco Croce, Maksym Andriushchenko, Vikash Sehwag +5
As a research community, we are still lacking a systematic understanding of the progress on adversarial robustness which often makes it hard to identify the most promising ideas in…
Understanding and Improving Fast Adversarial Training
Maksym Andriushchenko, Nicolas Flammarion
A recent line of work focused on making adversarial training computationally efficient for deep learning models. In particular, Wong et al. (2020) showed that -adversa…
Sparse-RS: a versatile framework for query-efficient sparse black-box adversarial attacks
Francesco Croce, Maksym Andriushchenko, Naman D. Singh +2
We propose a versatile framework based on random search, Sparse-RS, for score-based sparse targeted and untargeted attacks in the black-box setting. Sparse-RS does not rely on subs…
On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines
Marius Mosbach, Maksym Andriushchenko, Dietrich Klakow
Fine-tuning pre-trained transformer-based language models such as BERT has become a common practice dominating leaderboards across various NLP benchmarks. Despite the strong empiri…
Square Attack: a query-efficient black-box adversarial attack via random search
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion +1
We propose the Square Attack, a score-based black-box - and -adversarial attack that does not rely on local gradient information and thus is not affected by gradient…
Provably Robust Boosted Decision Stumps and Trees against Adversarial Attacks
Maksym Andriushchenko, Matthias Hein
The problem of adversarial robustness has been studied extensively for neural networks. However, for boosted decision trees and decision stumps there are almost no results, even th…
Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem
Matthias Hein, Maksym Andriushchenko, Julian Bitterwolf
Classifiers used in the wild, in particular for safety-critical systems, should not only have good generalization properties but also should know when they don't know, in particula…
Logit Pairing Methods Can Fool Gradient-Based Attacks
Marius Mosbach, Maksym Andriushchenko, Thomas Trost +2
Recently, Kannan et al. [2018] proposed several logit regularization methods to improve the adversarial robustness of classifiers. We show that the computationally fast methods the…