39 citations · 88 across the 63 of their papers we have counts for
66 papers
Causal Chain of Thought: Mitigating Attention Leakage via Context Isolation
Gabriele Dominici, Teague McMillan, Fazl Barez +2
Chain-of-thought (CoT) prompting elicits step-by-step rationales that appear to explain model predictions, but there is no guarantee these steps causally drive the final answer. In…
AIMO Interpretability Challenge
Michal Štefánik, Philipp Mondorf, Andreas Waldis +15
We propose the AIMO Interpretability Challenge, a competition on distinguishing robust from spurious reasoning in frontier mathematical language models based on the models' interna…
The 2026 Singapore Consensus on Global AI Safety Research Priorities
Stephen Casper, Oskar Galeev, Yoshua Bengio +117
Frontier AI capabilities and autonomy are advancing rapidly. A growing number of real-world incidents make a trusted AI ecosystem essential to embracing AI with confidence. The 202…
Pretraining Curricula Enable Selective Fine-tuning
Sebastian A. Bruijns, Jirko Rubruck, Mia H. Whitefield +3
Transformers follow implicit curricula whereby some tasks are learned before others. However, how explicit pretraining curricula influence learning, generalization, and the selecti…
The Capability Frontier: Benchmarks Miss 82% of Model Performance
Bradley Fowler, Ryan Smith, Daniel Thi Graviet +8
Existing benchmarks typically report accuracy for a single model on a single run. This systematically understates real-world LLM capabilities, particularly under heterogeneous data…
From Democracies to Autocracies: How AI Systems Enable Authoritarianism by Design
Jeba Sania, Marta Ziosi, Fazl Barez
AI-enabled authoritarianism is not confined to autocracies. In this paper, we provide greater transparency by investigating and mapping the lifecycles of six AI systems deployed in…
Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamics
Stella Biderman, Mohammad Aflah Khan, Niloofar Mireshghallah +3
What would it mean to have a scientific understanding of AI? Models are not static objects: they are snapshots of time-evolving processes shaped by data, objectives, architectures,…
Running the Gauntlet: Hard Agentic Tasks
Mykola Vysotskyi, Runqi Lin, Grzegorz Biziel +21
As agentic systems continue to evolve and are widely deployed in real-world scenarios, there is a growing demand to faithfully evaluate their capabilities. However, current benchma…
Make Mechanistic Interpretability Auditable: A Call to Develop Guidelines via Continuous Collaborative Reviewing
Michael Lan, Narmeen Fatimah Oozeer, Chaithanya Bandi +4
While mechanistic interpretability (MI) has produced important insights into neural network internals, the field has yet to establish a standardized system to audit experiments. As…
Interpretability Can Be Actionable
Hadas Orgad, Fazl Barez, Tal Haklay +9
Interpretability aims to explain the behavior of deep neural networks. Despite rapid growth, there is mounting concern that much of this work has not translated into practical impa…
Rigorous Interpretation Is a Form of Evaluation
Isabelle Lee, Emmy Liu, Cathy Jiao +4
Current machine learning models are evaluated through behavioral snapshots, with benchmark accuracies, win rates and outcome-based metrics. Model explanations and evaluations, howe…
Curveball Steering: The Right Direction To Steer Isn't Always Linear
Shivam Raval, Hae Jin Song, Linlin Wu +4
Activation steering is a widely used approach for controlling large language model (LLM) behavior by intervening on internal representations. Existing methods largely rely on the L…
AutoControl Arena: Synthesizing Executable Test Environments for Frontier AI Risk Evaluation
Changyi Li, Pengfei Lu, Xudong Pan +2
As Large Language Models (LLMs) evolve into autonomous agents, existing safety evaluations face a fundamental trade-off: manual benchmarks are costly, while LLM-based simulators ar…
Position: Token Taxes Can Mitigate AI's Economic Risks
Lucas Irwin, Tung-Yu Wu, Fazl Barez
AI-driven automation threatens to erode government tax bases, lower living standards, and disempower citizens--risks that mirror the 40-year stagnation of wages during the first in…
Old Habits Die Hard: How Conversational History Geometrically Traps LLMs
Adi Simhi, Fazl Barez, Martin Tutek +2
How does the conversational past of large language models (LLMs) influence their future performance? Recent work suggests that LLMs are affected by their conversational history in…
Same Answer, Different Representations: Hidden instability in VLMs
Farooq Ahmad Wani, Alessandro Suglia, Rohit Saxena +6
The robustness of Vision Language Models (VLMs) is commonly assessed through output-level invariance, implicitly assuming that stable predictions reflect stable multimodal processi…
Full-Stack Alignment: Co-Aligning AI and Institutions with Thick Models of Value
Joe Edelman, Tan Zhi-Xuan, Ryan Lowe +30
Beneficial societal outcomes cannot be guaranteed by aligning individual AI systems with the intentions of their operators or users. Even an AI system that is perfectly aligned to…
HACK: Hallucinations Along Certainty and Knowledge Axes
Adi Simhi, Jonathan Herzig, Itay Itzhak +7
Hallucinations in LLMs present a critical barrier to their reliable usage. Existing research usually categorizes hallucination by their external properties rather than by the LLMs'…
Chain-of-Thought Hijacking
Jianli Zhao, Tingchen Fu, Rylan Schaeffer +2
Large Reasoning Models (LRMs) improve task performance through extended inference-time reasoning. Although previous studies suggest that longer reasoning should lead to more robust…
VAL-Bench: Belief Consistency as a measure for Value Alignment in Language Models
Aman Gupta, Denny O'Shea, Fazl Barez
Large language models (LLMs) are increasingly being used for tasks where outputs shape human decisions, so it is critical to verify that their responses consistently reflect desire…
Embodied AI: Emerging Risks and Opportunities for Policy Action
Jared Perlo, Alexander Robey, Fazl Barez +2
The field of embodied AI (EAI) is rapidly advancing. Unlike virtual AI, EAI systems can exist in, learn from, reason about, and act in the physical world. With recent advances in A…
Beyond Linear Probes: Dynamic Safety Monitoring for Language Models
James Oldfield, Philip Torr, Ioannis Patras +2
Monitoring large language models' (LLMs) activations is an effective way to detect harmful requests before they lead to unsafe outputs. However, traditional safety monitors often r…
Query Circuits: Explaining How Language Models Answer User Prompts
Tung-Yu Wu, Fazl Barez
Explaining why a language model produces a particular output requires local, input-level explanations. Existing methods uncover global capability circuits (e.g., indirect object id…
Towards Understanding Subliminal Learning: When and How Hidden Biases Transfer
Simon Schrodi, Elias Kempf, Fazl Barez +1
Language models can transfer hidden biases during distillation. For example, a teacher that "likes owls" can make its student "like owls" too, even when the training data consists…