11 citations · 21 across the 9 of their papers we have counts for
9 papers
From token probabilities to calibrated confidence: An empirical study of mathematical question answering
Avery Ma, Lorne Schell, Vin Bhaskara +1
Confidence estimation for large language models (LLMs) aims to estimate the probability that a generated answer is correct, while calibration aligns these estimates with empirical…
StepJack: Benchmarking Computer-Use Agent Safety Against Multi-Step Indirect Prompt Injection
Zhuoxin Zhan, Akbar Rafiey, Avery Ma +2
Computer-use agents (CUAs) face a growing threat from indirect prompt injection, where adversarial instructions are planted in the environment such as web pages. In this paper, we…
PANDAS: Improving Many-shot Jailbreaking via Positive Affirmation, Negative Demonstration, and Adaptive Sampling
Avery Ma, Yangchen Pan, Amir-massoud Farahmand
Many-shot jailbreaking circumvents the safety alignment of LLMs by exploiting their ability to process long input sequences. To achieve this, the malicious target prompt is prefixe…
Improving Adversarial Transferability via Model Alignment
Avery Ma, Amir-massoud Farahmand, Yangchen Pan +2
Neural networks are susceptible to adversarial perturbations that are transferable across different models. In this paper, we introduce a novel model alignment technique aimed at i…
A Survey on Transferability of Adversarial Examples across Deep Neural Networks
Jindong Gu, Xiaojun Jia, Pau de Jorge +9
The emergence of Deep Neural Networks (DNNs) has revolutionized various domains by enabling the resolution of complex tasks spanning image recognition, natural language processing,…
Understanding the robustness difference between stochastic gradient descent and adaptive gradient methods
Avery Ma, Yangchen Pan, Amir-massoud Farahmand
Stochastic gradient descent (SGD) and adaptive gradient methods, such as Adam and RMSProp, have been widely used in training deep neural networks. We empirically show that while th…