16 citations · 63 across the 30 of their papers we have counts for
17 papers · 1 filter
PyINE: A Framework for Scalable Elicitation and Oversight via Code Execution
Pierre-Luc St-Charles, Alessandro Palmas, Damiano Fornasiere +5
Reasoning models can remain capable of solving a task while still defaulting to cheaper but misleading shortcuts. This creates a central oversight problem: when a model gives an an…
What if automating AI R&D triggers an intelligence explosion?
Alan Chan, Christoph Winter, Andrew Barto +19
In contrast to even a year ago, AI systems now write most of the code inside the companies that build them. As more of the AI research and development (R&D) pipeline is automated,…
AI Safety: Not Optional, Not Later
Qinghua Lu, Yoshua Bengio
Incidents show that AI safety failures often arise across multiple layers. We present a safety-by-design assurance architecture combining model-level supervision, such as Scientist…
How to Verify Probabilistic Consistency of Predictive Models
Orr Paradise, Oliver Richardson, Yoshua Bengio +1
When a probabilistic predictor answers many conditional-probability queries, are its answers self-consistent, and can this be verified in polynomial time? This problem is of intere…
Reducing Catastrophic Risk from AI with Systematic Monitoring and Evaluation of Rogue AI Progression
T. Bauer, W. P. Kegelmeyer, E. Begoli +15
This article presents a structured framework of behavioral indicators that may signal progression toward potentially catastrophic threats from artificial intelligence systems. We a…
Safety from Honesty in a Disinterested AI Predictor
Yoshua Bengio, Oliver Richardson, Tomáš Gavenčiak +13
As AI systems become more capable, training procedures that optimize for downstream outcomes risk introducing implicit agency: goal-directed behavior that designers never specified…
The 2026 Singapore Consensus on Global AI Safety Research Priorities
Stephen Casper, Oskar Galeev, Yoshua Bengio +117
Frontier AI capabilities and autonomy are advancing rapidly. A growing number of real-world incidents make a trusted AI ecosystem essential to embracing AI with confidence. The 202…
Open Technical Problems in Open-Weight AI Model Risk Management
Stephen Casper, Kyle O'Brien, Shayne Longpre +19
Frontier AI models with openly available weights are steadily becoming more powerful and widely adopted. However, compared to proprietary models, open-weight models pose different…
Towards Trustworthy Physical Intelligence: From Theory to Practice Across Life Cycle
Wang Yang, Yang Wang, Hongxuan Liu +45
Physical intelligence refers to intelligence systems that understand, reason about, and act in accordance with the physical world and its underlying laws, dynamics, and constraints…
CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs
Joachim Schaeffer, Thomas Jiralerspong, Alexander Panfilov +4
AI control protocols oversee untrusted models by monitoring their actions and modifying potentially unsafe steps, often using a trusted model. This partially tampers with the untru…
When 2D Tasks Meet 1D Serialization: On Serialization Friction in Structured Tasks
Chung-Hsiang Lo, Lu Li, Diji Yang +4
In the LLM era, many symbolic and structured problems are presented to models through 1D text serialization. Yet some such problems are natively two-dimensional: their relevant rel…
Noticing the Watcher: LLM Agents Can Infer CoT Monitoring from Blocking Feedback
Thomas Jiralerspong, Flemming Kondrup, Yoshua Bengio
Chain-of-thought (CoT) monitoring is proposed as a method for overseeing the internal reasoning of language-model agents. Prior work has shown that when models are explicitly infor…
The Silent Thought: Modeling Internal Cognition in Full-Duplex Spoken Dialogue Models via Latent Reasoning
Donghang Wu, Tianyu Zhang, Yuxin Li +4
During conversational interactions, humans subconsciously engage in concurrent thinking while listening to a speaker. Although this internal cognitive processing may not always man…
International AI Safety Report 2026
Yoshua Bengio, Stephen Clare, Carina Prunkl +89
The International AI Safety Report 2026 synthesises the current scientific evidence on the capabilities, emerging risks, and safety of general-purpose AI systems. The report series…
Frontier AI Auditing: Toward Rigorous Third-Party Assessment of Safety and Security Practices at Leading AI Companies
Miles Brundage, Noemi Dreksler, Aidan Homewood +45
We outline a vision for frontier AI auditing, which we define as rigorous third-party verification of frontier AI developers' safety and security claims, and evaluation of their sy…
When Benign Inputs Lead to Severe Harms: Eliciting Unsafe Unintended Behaviors of Computer-Use Agents
Jaylen Jones, Zhehao Zhang, Yuting Ning +6
Although computer-use agents (CUAs) hold significant potential to automate increasingly complex OS workflows, they can demonstrate unsafe unintended behaviors that deviate from exp…
In-Context Reinforcement Learning through Bayesian Fusion of Context and Value Prior
Anaïs Berkes, Vincent Taboga, Donna Vakalis +2
In-context reinforcement learning (ICRL) promises fast adaptation to unseen environments without parameter updates, but current methods either cannot improve beyond the training di…