11 citations · 21 across the 9 of their papers we have counts for
9 papers
From token probabilities to calibrated confidence: An empirical study of mathematical question answering
Avery Ma, Lorne Schell, Vin Bhaskara +1
Confidence estimation for large language models (LLMs) aims to estimate the probability that a generated answer is correct, while calibration aligns these estimates with empirical…
StepJack: Benchmarking Computer-Use Agent Safety Against Multi-Step Indirect Prompt Injection
Zhuoxin Zhan, Akbar Rafiey, Avery Ma +2
Computer-use agents (CUAs) face a growing threat from indirect prompt injection, where adversarial instructions are planted in the environment such as web pages. In this paper, we…
PANDAS: Improving Many-shot Jailbreaking via Positive Affirmation, Negative Demonstration, and Adaptive Sampling
Avery Ma, Yangchen Pan, Amir-massoud Farahmand
Many-shot jailbreaking circumvents the safety alignment of LLMs by exploiting their ability to process long input sequences. To achieve this, the malicious target prompt is prefixe…
Improving Adversarial Transferability via Model Alignment
Avery Ma, Amir-massoud Farahmand, Yangchen Pan +2
Neural networks are susceptible to adversarial perturbations that are transferable across different models. In this paper, we introduce a novel model alignment technique aimed at i…
A Survey on Transferability of Adversarial Examples across Deep Neural Networks
Jindong Gu, Xiaojun Jia, Pau de Jorge +9
The emergence of Deep Neural Networks (DNNs) has revolutionized various domains by enabling the resolution of complex tasks spanning image recognition, natural language processing,…
Understanding the robustness difference between stochastic gradient descent and adaptive gradient methods
Avery Ma, Yangchen Pan, Amir-massoud Farahmand
Stochastic gradient descent (SGD) and adaptive gradient methods, such as Adam and RMSProp, have been widely used in training deep neural networks. We empirically show that while th…
SAGE: Saliency-Guided Mixup with Optimal Rearrangements
Avery Ma, Nikita Dvornik, Ran Zhang +3
Data augmentation is a key element for training accurate models by reducing overfitting and improving generalization. For image classification, the most popular data augmentation t…
Improving Hierarchical Adversarial Robustness of Deep Neural Networks
Avery Ma, Aladin Virmaux, Kevin Scaman +1
Do all adversarial examples have the same consequences? An autonomous driving system misclassifying a pedestrian as a car may induce a far more dangerous -- and even potentially le…
SOAR: Second-Order Adversarial Regularization
Avery Ma, Fartash Faghri, Nicolas Papernot +1
Adversarial training is a common approach to improving the robustness of deep neural networks against adversarial examples. In this work, we propose a novel regularization approach…