10 papers
A Survey of Adversarial Efficiency Degradation for Vision Transformer by Exploiting Input-adaptive Optimization
Anadi Goyal, Nandish Chattopadhyay, Anupam Chattopadhyay +1
Vision Transformers (ViTs) increasingly rely on input-adaptive inference, such as token pruning and early halting, to meet energy and latency budgets. This survey examines a recent…
MOAT: Model-Agnostic Randomized Transformations for preventing Efficiency Degradation Attacks on ViTs
Anadi Goyal, Nandish Chattopadhyay, Chandan Karfa +2
To adopt the Vision Transformers (ViTs) in resource-constrained environment, token pruning is widely used to reduce computational cost without impacting accuracy. However, adversar…
On the Limits of Support-Preserving Alignment and Bounded Filtering
Aryan Dutt, Rui Mao, Anupam Chattopadhyay
We study whether alignment schemes that reshape a base model's output distribution, combined with bounded safety filters, can drive the probability of harmful behavior to zero in m…
Vaporizer: Breaking Watermarking Schemes for Large Language Model Outputs
Jonathan Hong Jin Ng, Anh Tu Ngo, Anupam Chattopadhyay
In this paper, we investigate the recent state-of-the-art schemes for watermarking large language models (LLMs) outputs. These techniques are claimed to be robust, scalable and pro…
NutVLM: A Self-Adaptive Defense Framework against Full-Dimension Attacks for Vision Language Models in Autonomous Driving
Xiaoxu Peng, Dong Zhou, Jianwen Zhang +3
Vision Language Models (VLMs) have advanced perception in autonomous driving (AD), but they remain vulnerable to adversarial threats. These risks range from localized physical patc…
STRAP-ViT: Segregated Tokens with Randomized -- Transformations for Defense against Adversarial Patches in ViTs
Nandish Chattopadhyay, Anadi Goyal, Chandan Karfa +1
Adversarial patches are physically realizable localized noise, which are able to hijack Vision Transformers (ViT) self-attention, pulling focus toward a small, high-contrast region…