158 citations · 389 across the 81 of their papers we have counts for
88 papers
Securing Computer-Use Agents Against Branch Steering Attacks
Giulio Zingrillo, Hanna Foerster, Ilia Shumailov +2
Modern Computer Use Agents (CUAs) directly interact with graphical user interfaces and execute third-party web tools, exposing them to indirect prompt injection across every render…
AgentPerfBench: A Benchmarking and Evaluation Suite for Inference Performance of Agentic LLMs
Cheuk Hang Lau, Zeyu Cao, Kevin Wong Cheuk Yin +6
The optimization of LLM serving engines, such as vLLM and SGLang, is largely benchmark-driven: optimizations, scheduling policies, hardware and system designs are all selected base…
CodeMechanic: Bug-Property-Guided Program Mitigation
Han Zheng, Rafaila Galanopoulou, Ilia Shumailov +4
Automated testing discovers vulnerabilities faster than developers can investigate and repair them, leaving an interval in which known memory corruptions remain exploitable. End- t…
DumpsterCluster: From Dumpster Diving to Serving LLaMA-70B on $60 GPUs
Zeyu Cao, Xuan Guo, Cheng Zhang +3
As AI datacenters retire functional GPUs, vast quantities of still capable accelerators enter secondary markets. This paper investigates whether these retired GPUs can find a produ…
GitInject: Real-World Prompt Injection Attacks in AI-Powered CI/CD Pipelines
Jafar Isbarov, Umid Suleymanov, Ilia Shumailov +1
AI agents are increasingly embedded in continuous integration and continuous delivery (CI/CD) workflows to autonomously review pull requests, triage issues, and maintain codebases.…
Honeyval: A Comprehensive Evaluation Framework for LLM-powered HTTP Honeypots
Mark Vero, Fabian Kaczmarczyck, Ivan Petrov +6
Honeypots are decoy systems mimicking real system components designed to defend against cyber attacks. Recently, LLMs increasingly serve as simulation backbones for honeypots. They…
Quantamination: Dynamic Quantization Leaks Your Data Across the Batch
Hanna Foerster, Ilia Shumailov, Cheng Zhang +3
Dynamic quantization emerged as a practical approach to increase the utilization and efficiency of the machine learning serving flow. Unlike static quantization, which applies quan…
Kraken: Higher-order EM Side-Channel Attacks on DNNs in Near and Far Field
Peter Horvath, Ilia Shumailov, Lukasz Chmielewski +2
The multi-million dollar investment required for modern machine learning (ML) has made large ML models a prime target for theft. In response, the field of model stealing has emerge…
Thought-Transfer: Indirect Targeted Poisoning Attacks on Chain-of-Thought Reasoning Models
Harsh Chaudhari, Ethan Rathbun, Hanna Foerster +5
Chain-of-Thought (CoT) reasoning has emerged as a powerful technique for enhancing large language models' capabilities by generating intermediate reasoning steps for complex tasks.…
CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents
Hanna Foerster, Tom Blanchard, Kristina Nikolić +6
AI agents are vulnerable to prompt injection attacks, where malicious content hijacks agent behavior. Among proposed defenses, architectural isolation provides the strongest guaran…
Iterative Deployment Improves Planning Skills in LLMs
Augusto B. Corrêa, Yoav Gelberg, Luckeciano C. Melo +3
We show that iterative deployment of large language models (LLMs), each fine-tuned on data carefully curated by users from the previous models' deployment, can significantly change…
ceLLMate: Sandboxing Browser AI Agents
Luoxi Meng, Henry Feng, Ilia Shumailov +1
Browser-using agents (BUAs) are an emerging class of AI agents that interact with web browsers in human-like ways, including clicking, scrolling, filling forms, and navigating acro…
Extracting alignment data in open models
Federico Barbero, Xiangming Gu, Christopher A. Choquette-Choo +6
In this work, we show that it is possible to extract significant amounts of alignment training data from a post-trained model -- useful to steer the model to improve certain capabi…
SynthID-Image: Image watermarking at internet scale
Sven Gowal, Rudy Bunel, Florian Stimberg +23
We introduce SynthID-Image, a deep learning-based system for invisibly watermarking AI-generated imagery. This paper documents the technical desiderata, threat models, and practica…
The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections
Milad Nasr, Nicholas Carlini, Chawin Sitawarin +11
How should we evaluate the robustness of language model defenses? Current defenses against jailbreaks and prompt injections (which aim to prevent an attacker from eliciting harmful…
Soft Instruction De-escalation Defense
Nils Philipp Walter, Chawin Sitawarin, Jamie Hayes +2
Large Language Models (LLMs) are increasingly deployed in agentic systems that interact with an external environment; this makes them susceptible to prompt injections when dealing…
Reasoning Introduces New Poisoning Attacks Yet Makes Them More Complicated
Hanna Foerster, Ilia Shumailov, Yiren Zhao +4
Early research into data poisoning attacks against Large Language Models (LLMs) demonstrated the ease with which backdoors could be injected. More recent LLMs add step-by-step reas…
Beyond Laplace and Gaussian: Exploring the Generalized Gaussian Mechanism for Private Machine Learning
Roy Rinberg, Ilia Shumailov, Vikrant Singhal +2
Differential privacy (DP) is obtained by randomizing a data analysis algorithm, which necessarily introduces a tradeoff between its utility and privacy. Many DP mechanisms are buil…
Machine Learning Models Have a Supply Chain Problem
Sarah Meiklejohn, Hayden Blauzvern, Mihai Maruseac +3
Powerful machine learning (ML) models are now readily available online, which creates exciting possibilities for users who lack the deep technical expertise or substantial computin…
Fixing 7,400 Bugs for 1$: Cheap Crash-Site Program Repair
Han Zheng, Ilia Shumailov, Tianqi Fan +2
The rapid advancement of bug-finding techniques has led to the discovery of more vulnerabilities than developers can reasonably fix, creating an urgent need for effective Automated…
Lessons from Defending Gemini Against Indirect Prompt Injections
Chongyang Shi, Sharon Lin, Shuang Song +11
Gemini is increasingly used to perform tasks on behalf of users, where function-calling and tool-use capabilities enable the model to access user data. Some tools, however, require…
Cascading Adversarial Bias from Injection to Distillation in Language Models
Harsh Chaudhari, Jamie Hayes, Matthew Jagielski +3
Model distillation has become essential for creating smaller, deployable language models that retain larger system capabilities. However, widespread deployment raises concerns abou…
Exploring the limits of strong membership inference attacks on large language models
Jamie Hayes, Ilia Shumailov, Christopher A. Choquette-Choo +13
State-of-the-art membership inference attacks (MIAs) typically require training many reference models, making it difficult to scale these attacks to large pre-trained language mode…
Architectural Backdoors for Within-Batch Data Stealing and Model Inference Manipulation
Nicolas Küchler, Ivan Petrov, Conrad Grobler +1
For nearly a decade the academic community has investigated backdoors in neural networks, primarily focusing on classification tasks where adversaries manipulate the model predicti…