3 citations · 3 across the 5 of their papers we have counts for
5 papers
Refusal geometry reflects refusal training: diverse refusal prefixes can raise stable rank and weaken refusal vector ablation attacks
Andrey Labunets
Refusal training protects AI models from jailbreaks by training models to decline unsafe queries, reducing the risk of misuse. Recent work finds that refusal behavior in aligned la…
Agent Security is a Systems Problem
Mihai Christodorescu, Earlence Fernandes, Ashish Hooda +11
We take the position that agent security must be approached as a systems problem: the AI model powering the agent must be treated as an untrusted component, and security invariants…
May I have your Attention? Breaking Fine-Tuning based Prompt Injection Defenses using Architecture-Aware Attacks
Nishit V. Pandya, Andrey Labunets, Sicun Gao +1
A popular class of defenses against prompt injection attacks on large language models (LLMs) relies on fine-tuning to separate instructions and data, so that the LLM does not follo…
Fun-tuning: Characterizing the Vulnerability of Proprietary LLMs to Optimization-based Prompt Injection Attacks via the Fine-Tuning Interface
Andrey Labunets, Nishit V. Pandya, Ashish Hooda +2
We surface a new threat to closed-weight Large Language Models (LLMs) that enables an attacker to compute optimization-based prompt injections. Specifically, we characterize how an…
Re-purposing Perceptual Hashing based Client Side Scanning for Physical Surveillance
Ashish Hooda, Andrey Labunets, Tadayoshi Kohno +1
Content scanning systems employ perceptual hashing algorithms to scan user content for illegal material, such as child pornography or terrorist recruitment flyers. Perceptual hashi…