239 citations · 271 across the 19 of their papers we have counts for
23 papers
What the Model Cannot See: Covert and Provable Text Invisibility in LLMs
Mark Russinovich
Text wrapped in a trigger can be made behaviorally invisible to a language model: omitted from summaries and quotations, ignored in question answering, and left unflagged by downst…
CounterSteer: Suppressing Indirect Prompt Injection with Activation Steering
Mark Russinovich
Indirect prompt injection makes an LLM agent treat untrusted retrieved text as instructions. We present CounterSteer, an inference-time defense that suppresses this behavior inside…
Divide, Consult, Conquer: Capability Laundering Through Aligned LLMs
Mark Russinovich, Blake Bullwinkel, Giorgio Severi +2
Language model safety is typically evaluated one interaction at a time. We show that a weaker, unaligned model can split a harmful task into benign-looking subproblems, consult a s…
SpecGuard: Inference-Time Backdoor Detection For Free
Rui Wen, Ahmed Salem, Andrew Paverd +2
Large language models are often fine-tuned, shared, or downloaded from third parties, so a deployed model may carry a hidden backdoor that behaves normally on benign inputs but swi…
Fool's Gold: Defensive Deception Against Safety-Removal Attacks on Open-Weight Models
Mark Russinovich
Safety alignment in open-weight language models is trivially removable: abliteration projects a refusal-mediating direction out of the weights in minutes, and no release-time defen…
Phantom References: Hallucinated Citations That Survive Peer Review at Top-Tier Conferences
Mark Russinovich, Ram Shankar Siva Kumar, Ahmed Salem
Large language models can generate polished scientific text that includes unsupported claims, allowing hallucinations to enter the archival record. Assessing this risk via technica…
Learning to Attack and Defend: Adaptive Red Teaming of Language Models via GRPO
Blake Bullwinkel, Eugenia Kim, Amanda Minnich +1
Language model safety must continually adapt to evolving attacks. Recent works have demonstrated that reinforcement learning can be used to train stronger attacker and defender mod…
MetaBackdoor: Exploiting Positional Encoding as a Backdoor Attack Surface in LLMs
Rui Wen, Mark Russinovich, Andrew Paverd +2
Backdoor attacks pose a serious security threat to large language models (LLMs), which are increasingly deployed as general-purpose assistants in safety- and privacy-critical appli…
Optimizing Agent Planning for Security and Autonomy
Aashish Kolluri, Rishi Sharma, Manuel Costa +5
Indirect prompt injection attacks threaten AI agents that execute consequential actions, motivating deterministic system-level defenses. Such defenses can provably block unsafe act…
GRP-Obliteration: Unaligning LLMs With a Single Unlabeled Prompt
Mark Russinovich, Yanan Cai, Keegan Hines +3
Safety alignment is only as robust as its weakest failure mode. Despite extensive work on safety post-training, it has been shown that models can be readily unaligned through post-…
A Representation Engineering Perspective on the Effectiveness of Multi-Turn Jailbreaks
Blake Bullwinkel, Mark Russinovich, Ahmed Salem +8
Recent research has demonstrated that state-of-the-art LLMs and defenses remain susceptible to multi-turn jailbreak attacks. These attacks require only closed-box model access and…
LogiPlan: A Structured Benchmark for Logical Planning and Relational Reasoning in LLMs
Yanan Cai, Ahmed Salem, Besmira Nushi +1
We introduce LogiPlan, a novel benchmark designed to evaluate the capabilities of large language models (LLMs) in logical planning and reasoning over complex relational structures.…
LLMail-Inject: A Dataset from a Realistic Adaptive Prompt Injection Challenge
Sahar Abdelnabi, Aideen Fay, Ahmed Salem +22
Indirect prompt injection attacks exploit a fundamental weakness of LLMs: executing unauthorized tasks from untrusted sources. Despite the practical significance of this attack vec…
Securing AI Agents with Information-Flow Control
Manuel Costa, Boris Köpf, Aashish Kolluri +6
As AI agents become increasingly autonomous and capable, ensuring their security against vulnerabilities such as prompt injection becomes critical. This paper explores the use of i…
Jailbreaking is (Mostly) Simpler Than You Think
Mark Russinovich, Ahmed Salem
We introduce the Context Compliance Attack (CCA), a novel, optimization-free method for bypassing AI safety mechanisms. Unlike current approaches -- which rely on complex prompt en…
Obliviate: Efficient Unmemorization for Protecting Intellectual Property in Large Language Models
Mark Russinovich, Ahmed Salem
Recent copyright agreements between AI companies and content creators underscore the need for fine-grained control over language models' ability to reproduce copyrighted text. Exis…
Lessons From Red Teaming 100 Generative AI Products
Blake Bullwinkel, Amanda Minnich, Shiven Chawla +23
In recent years, AI red teaming has emerged as a practice for probing the safety and security of generative AI systems. Due to the nascency of the field, there are many open questi…
Hey, That's My Model! Introducing Chain & Hash, An LLM Fingerprinting Technique
Mark Russinovich, Yanan Cai, Ahmed Salem
Growing concerns over the theft and misuse of Large Language Models (LLMs) underscore the need for effective fingerprinting to link a model to its original version and detect misus…
Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack
Mark Russinovich, Ahmed Salem, Ronen Eldan
Large Language Models (LLMs) have risen significantly in popularity and are increasingly being adopted across multiple applications. These LLMs are heavily aligned to resist engagi…
Confidential Consortium Framework: Secure Multiparty Applications with Confidentiality, Integrity, and High Availability
Heidi Howard, Fritz Alder, Edward Ashton +12
Confidentiality, integrity protection, and high availability, abbreviated to CIA, are essential properties for trustworthy data systems. The rise of cloud computing and the growing…
Who's Harry Potter? Approximate Unlearning in LLMs
Ronen Eldan, Mark Russinovich
Large language models (LLMs) are trained on massive internet corpora that often contain copyrighted content. This poses legal and ethical challenges for the developers and users of…
Singularity: Planet-Scale, Preemptive and Elastic Scheduling of AI Workloads
Dharma Shukla, Muthian Sivathanu, Srinidhi Viswanatha +23
Lowering costs by driving high utilization across deep learning workloads is a crucial lever for cloud providers. We present Singularity, Microsoft's globally distributed schedulin…
Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider
Mohammad Shahrad, Rodrigo Fonseca, Íñigo Goiri +7
Function as a Service (FaaS) has been gaining popularity as a way to deploy computations to serverless backends in the cloud. This paradigm shifts the complexity of allocating and…