3 citations · 3 across the 9 of their papers we have counts for
4 papers · 1 filter
Character Training for Risk-Averse Agents
Arav Dhoot, Punya Syon Pandey, Jamie Johnson +3
Risk aversion in resources could prevent misaligned AI agents from causing catastrophic harm. Misaligned but risk-averse agents would tend to favor safer strategies like making dea…
Don't Inoculate Everything: Stratified Inoculation Prompting Narrows Backdoor Triggers and Preserves Desired Traits
Kajetan Dymkiewicz, Tim Farrelly, Adam Práda +8
Supervised fine-tuning can teach language models undesired behaviours alongside desired ones. Inoculation prompting (IP) aims to limit unwanted generalisation by requesting the und…
Inoculation Adapters: Improved Selective Generalization of Capabilities with Fewer Surprising Backdoors
Maxime Riché, Daniel Tan, Vili Kohonen +1
Inoculation prompting is a selective-generalization technique used against Emergent Misalignment. We introduce inoculation adapters (IA), a family of methods that similarly reduce…
Spilling the Beans: Teaching LLMs to Self-Report Their Hidden Objectives
Chloe Li, Mary Phuong, Daniel Tan
As AI systems become more capable of complex agentic tasks, they also become more capable of pursuing undesirable objectives and causing harm. Previous work has attempted to catch…