6 papers
Stealing Reasoning Traces from Proprietary LLM APIs
Alexander Panfilov, David Schmotz, Ilia Shumailov +5
Leading large language model providers now conceal their models' step-by-step reasoning, or chain-of-thought, to protect intellectual property and limit information leakage. Rather…
Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols
Mikhail Terekhov, Alexander Panfilov, Daniil Dzenhaliou +4
AI control protocols serve as a defense mechanism to stop untrusted LLM agents from causing harm in autonomous settings. Prior work treats this as a security problem, stress testin…
Capability-Based Scaling Trends for LLM-Based Red-Teaming
Alexander Panfilov, Paul Kassianik, Maksym Andriushchenko +1
As large language models grow in capability and agency, identifying vulnerabilities through red-teaming becomes vital for safe deployment. However, traditional prompt-engineering a…
Strategic Dishonesty Can Undermine AI Safety Evaluations of Frontier LLMs
Alexander Panfilov, Evgenii Kortukov, Kristina NikoliÄ +6
Large language model (LLM) developers aim for their models to be honest, helpful, and harmless. However, when faced with malicious requests, models are trained to refuse, sacrifici…
Critical Influence of Overparameterization on Sharpness-aware Minimization
Sungbin Shin, Dongyeop Lee, Maksym Andriushchenko +1
Sharpness-Aware Minimization (SAM) has attracted considerable attention for its effectiveness in improving generalization in deep neural network training by explicitly minimizing s…
Exploring Memorization and Copyright Violation in Frontier LLMs: A Study of the New York Times v. OpenAI 2023 Lawsuit
Joshua Freeman, Chloe Rippe, Edoardo Debenedetti +1
Copyright infringement in frontier LLMs has received much attention recently due to the New York Times v. OpenAI lawsuit, filed in December 2023. The New York Times claims that GPT…