39 citations · 88 across the 63 of their papers we have counts for
17 papers · 1 filter
AIMO Interpretability Challenge
Michal Štefánik, Philipp Mondorf, Andreas Waldis +15
We propose the AIMO Interpretability Challenge, a competition on distinguishing robust from spurious reasoning in frontier mathematical language models based on the models' interna…
The Capability Frontier: Benchmarks Miss 82% of Model Performance
Bradley Fowler, Ryan Smith, Daniel Thi Graviet +8
Existing benchmarks typically report accuracy for a single model on a single run. This systematically understates real-world LLM capabilities, particularly under heterogeneous data…
Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamics
Stella Biderman, Mohammad Aflah Khan, Niloofar Mireshghallah +3
What would it mean to have a scientific understanding of AI? Models are not static objects: they are snapshots of time-evolving processes shaped by data, objectives, architectures,…
Curveball Steering: The Right Direction To Steer Isn't Always Linear
Shivam Raval, Hae Jin Song, Linlin Wu +4
Activation steering is a widely used approach for controlling large language model (LLM) behavior by intervening on internal representations. Existing methods largely rely on the L…
AutoControl Arena: Synthesizing Executable Test Environments for Frontier AI Risk Evaluation
Changyi Li, Pengfei Lu, Xudong Pan +2
As Large Language Models (LLMs) evolve into autonomous agents, existing safety evaluations face a fundamental trade-off: manual benchmarks are costly, while LLM-based simulators ar…
Same Answer, Different Representations: Hidden instability in VLMs
Farooq Ahmad Wani, Alessandro Suglia, Rohit Saxena +6
The robustness of Vision Language Models (VLMs) is commonly assessed through output-level invariance, implicitly assuming that stable predictions reflect stable multimodal processi…
Chain-of-Thought Hijacking
Jianli Zhao, Tingchen Fu, Rylan Schaeffer +2
Large Reasoning Models (LRMs) improve task performance through extended inference-time reasoning. Although previous studies suggest that longer reasoning should lead to more robust…
VAL-Bench: Belief Consistency as a measure for Value Alignment in Language Models
Aman Gupta, Denny O'Shea, Fazl Barez
Large language models (LLMs) are increasingly being used for tasks where outputs shape human decisions, so it is critical to verify that their responses consistently reflect desire…
Query Circuits: Explaining How Language Models Answer User Prompts
Tung-Yu Wu, Fazl Barez
Explaining why a language model produces a particular output requires local, input-level explanations. Existing methods uncover global capability circuits (e.g., indirect object id…
Establishing Best Practices for Building Rigorous Agentic Benchmarks
Yuxuan Zhu, Tengjun Jin, Yada Pruksachatkun +22
Benchmarks are essential for quantitatively tracking progress in AI. As AI agents become increasingly capable, researchers and practitioners have introduced agentic benchmarks to e…
The Singapore Consensus on Global AI Safety Research Priorities
Yoshua Bengio, Tegan Maharaj, Luke Ong +84
Rapidly improving AI capabilities and autonomy hold significant promise of transformation, but are also driving vigorous debate on how to ensure that AI is safe, i.e., trustworthy,…
Beyond Black-Box Obfuscation: Mechanistic Analysis and Defense of White-Box Monitors
Maheep Chaudhary, Fazl Barez
White-box monitoring is increasingly adopted as an auditing tool as Large Language Models (LLMs) are deployed in daily operations to ensure safe model behavior. However, white-box…
Rethinking AI Cultural Alignment
Michal Bravansky, Filip Trhlik, Fazl Barez
As general-purpose artificial intelligence (AI) systems become increasingly integrated with diverse human communities, cultural alignment has emerged as a crucial element in their…
Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models
Carson Denison, Monte MacDiarmid, Fazl Barez +11
In reinforcement learning, specification gaming occurs when AI systems learn undesired behaviors that are highly rewarded due to misspecified training goals. Specification gaming c…
Large Language Models Relearn Removed Concepts
Michelle Lo, Shay B. Cohen, Fazl Barez
Advances in model editing through neuron pruning hold promise for removing undesirable concepts from large language models. However, it remains unclear whether models have the capa…
Measuring Value Alignment
Fazl Barez, Philip Torr
As artificial intelligence (AI) systems become increasingly integrated into various domains, ensuring that they align with human values becomes critical. This paper introduces a no…
AI Systems of Concern
Kayla Matteucci, Shahar Avin, Fazl Barez +1
Concerns around future dangers from advanced AI often centre on systems hypothesised to have intrinsic characteristics such as agent-like behaviour, strategic awareness, and long-r…