13 papers
Scalable Weak-to-Strong Verifier Hardening for Agent Environments
Chong Zeng, Ziqian Zhong, Kexun Zhang +5
Agent environments are essential for large language model (LLM) development. They use programmatic verifiers to turn complex outcomes into rewards. Verifiers can be attacked into a…
Hardening Agent Benchmarks with Adversarial Hacker-Fixer Loops
Ziqian Zhong, Ivgeni Segal, Ivan Bercovich +3
Agent benchmarks score submissions with outcome verifiers that are typically hand-written and brittle, leaving them open to reward hacking. We audit 1,968 tasks across five termina…
Self-Trained Verification for Training- and Test-Time Self-Improvement
Chen Henry Wu, Aditi Raghunathan
Self-improvement at scale has been a longstanding goal for reasoning models, and there are two natural places to do it: at test time, through verification-refinement (V-R) loops; a…
Base Models Look Human To AI Detectors
Yixuan Even Xu, Ziqian Zhong, Aditi Raghunathan +2
As AI-generated text enters the real-world at scale, institutions increasingly use commercial AI-text detectors, especially in education and academic-integrity workflows. We report…
Understanding and Mitigating Premature Confidence for Better LLM Reasoning
Jingchu Gai, Guanning Zeng, Christina Baek +4
Long chains of thought (CoT) from current language models frequently contain logical gaps and unjustified leaps, limiting the gains from additional test-time compute. Improving rea…
Terminal Wrench: A Dataset of 331 Reward-Hackable Environments and 3,632 Exploit Trajectories
Ivan Bercovich, Ivgeni Segal, Kexun Zhang +3
We release Terminal Wrench, a subset of 331 terminal-agent benchmark environments, copied from the popular open benchmarks that are demonstrably reward-hackable. The data set inclu…
Hodoscope: Unsupervised Monitoring for AI Misbehaviors
Ziqian Zhong, Shashwat Saxena, Aditi Raghunathan
Existing approaches to monitoring AI agents rely on supervised evaluation: human-written rules or LLM-based judges that check for known failure modes. However, novel misbehaviors m…
Pando: Do Interpretability Methods Work When Models Won't Explain Themselves?
Ziqian Zhong, Aashiq Muhamed, Mona T. Diab +2
Mechanistic interpretability is often motivated for alignment auditing, where a model's verbal explanations can be absent, incomplete, or misleading. Yet many evaluations do not co…
Jailbreaking in the Haystack
Rishi Rajesh Shah, Chen Henry Wu, Shashwat Saxena +3
Recent advances in long-context language models (LMs) have enabled million-token inputs, expanding their capabilities across complex tasks like computer-use agents. Yet, the safety…
ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases
Ziqian Zhong, Aditi Raghunathan, Nicholas Carlini
The tendency to find and exploit "shortcuts" to complete tasks poses significant risks for reliable assessment and deployment of large language models (LLMs). For example, an LLM a…
Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMs
Ziqian Zhong, Aditi Raghunathan
The releases of powerful open-weight large language models (LLMs) are often not accompanied by access to their full training data. Existing interpretability methods, particularly t…
Vulnerability of Text-Matching in ML/AI Conference Reviewer Assignments to Collusions
Jhih-Yi Hsieh, Aditi Raghunathan, Nihar B. Shah
In the peer review process of top-tier machine learning (ML) and artificial intelligence (AI) conferences, reviewers are assigned to papers through automated methods. These assignm…
Scaling Laws for Precision
Tanishq Kumar, Zachary Ankner, Benjamin F. Spector +6
Low precision training and inference affect both the quality and cost of language models, but current scaling laws do not account for this. In this work, we devise "precision-aware…