9 papers
Overthinking: Amplifying Reasoning Weights to Extract Learned Secrets
Jack Hopkins, Dipika Khullar, Fabien Roger
Black box auditing of language models is an essential pre-deployment tool, but it may miss subtle forms of misalignment and hidden information. To better elicit hidden information…
Narrow Secret Loyalty Dodges Black-Box Audits
Alfie Lamerton, Fabien Roger
Recent work identifies secret loyalties as a distinct threat from standard backdoors. A secret loyalty causes a model to covertly advance the interests of a specific principal whil…
SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors
Elle Najt, Colin Toft, Tyler Tracy +2
Since autonomous coding agents generate complex behaviors at high-volume, we may want to use other LLMs to monitor actions to reduce the risk from dangerous misaligned behavior. To…
Classifier Context Rot: Monitor Performance Degrades with Context Length
Sam Martin, Fabien Roger
Monitoring coding agents for dangerous behavior using language models requires classifying transcripts that often exceed 500K tokens, but prior agent monitoring benchmarks rarely c…
How Useful Is Cross-Domain Generalization for Training LLM Monitors?
Sam Martin, Fabien Roger
Using prompted language models as classifiers enables classification in domains with limited training data, but misses some of the robustness and performance benefits that fine-tun…
Self-Attribution Bias: When AI Monitors Go Easy on Themselves
Dipika Khullar, Jack Hopkins, Rowan Wang +1
Agentic systems increasingly rely on language models to monitor their own behavior. For example, coding agents may self critique generated code for pull request approval or assess…