Publications (112)
Quantifying Local Specialization in Deep Neural Networks
Shlomi Hod, Daniel Filan, Stephen Casper +2
A neural network is locally specialized to the extent that parts of its computational graph (i.e. structure) can be abstractly represented as performing some comprehensible sub-tas…
Automated Construction of Sparse Bayesian Networks from Unstructured Probabilistic Models and Domain Information
Sampath Srinivas, Stuart Russell, Alice M. Agogino
An algorithm for automated construction of a sparse Bayesian network given an unstructured probabilistic model and causal domain information from an expert has been developed and i…
The Off-Switch Game
Dylan Hadfield-Menell, Anca Dragan, Pieter Abbeel +1
It is clear that one of the primary tools we can use to mitigate the potential risk from a misbehaving AI system is the ability to turn the system off. As the capabilities of AI sy…
RAPID: A Reachable Anytime Planner for Imprecisely-sensed Domains
Emma Brunskill, Stuart Russell
Despite the intractability of generic optimal partially observable Markov decision process planning, there exist important problems that have highly structured models. Previous res…
International AI Safety Report 2025: First Key Update: Capabilities and Risk Implications
Yoshua Bengio, Stephen Clare, Carina Prunkl +70
Since the publication of the first International AI Safety Report, AI capabilities have continued to improve across key domains. New training techniques that teach AI systems to re…
DERAIL: Diagnostic Environments for Reward And Imitation Learning
Pedro Freire, Adam Gleave, Sam Toyer +1
The objective of many real-world tasks is complex and difficult to procedurally specify. This makes it necessary to use reward or imitation learning algorithms to infer a reward or…
The Effective Horizon Explains Deep RL Performance in Stochastic Environments
Cassidy Laidlaw, Banghua Zhu, Stuart Russell +1
Reinforcement learning (RL) theory has largely focused on proving minimax sample complexity bounds. These require strategic exploration algorithms that use relatively limited funct…
Bridging Offline Reinforcement Learning and Imitation Learning: A Tale of Pessimism
Paria Rashidinejad, Banghua Zhu, Cong Ma +2
Offline (or batch) reinforcement learning (RL) algorithms seek to learn an optimal policy from a fixed dataset without active data collection. Based on the composition of the offli…
Invariance in Policy Optimisation and Partial Identifiability in Reward Learning
Joar Skalse, Matthew Farrugia-Roberts, Stuart Russell +2
It is often very challenging to manually design reward functions for complex, real-world tasks. To solve this, one can instead use reward learning to infer a reward function from d…
Learning the Preferences of a Learning Agent
Karim Abdel Sadek, Mark Bedaywi, Rhys Gould +1
For AI systems to be useful to humans, they must understand and act in accordance with our values and preferences. Since specifying preferences is a hard task, inverse reinforcemen…
When Your AIs Deceive You: Challenges of Partial Observability in Reinforcement Learning from Human Feedback
Leon Lang, Davis Foote, Stuart Russell +3
Past analyses of reinforcement learning from human feedback (RLHF) assume that the human evaluators fully observe the environment. What happens when human feedback is based only on…
Generalization or Hallucination? Understanding Out-of-Context Reasoning in Transformers
Yixiao Huang, Hanlin Zhu, Tianyu Guo +5
Large language models (LLMs) can acquire new knowledge through fine-tuning, but this process exhibits a puzzling duality: models can generalize remarkably from new facts, yet are a…
Clusterability in Neural Networks
Daniel Filan, Stephen Casper, Shlomi Hod +3
The learned weights of a neural network have often been considered devoid of scrutable internal structure. In this paper, however, we look for structure in the form of clusterabili…
A temporally abstracted Viterbi algorithm
Shaunak Chatterjee, Stuart Russell
Hierarchical problem abstraction, when applicable, may offer exponential reductions in computational complexity. Previous work on coarse-to-fine dynamic programming (CFDP) has demo…
Statistical Guarantees for Offline Domain Randomization
Arnaud Fickinger, Abderrahim Bendahi, Stuart Russell
Reinforcement-learning (RL) agents often struggle when deployed from simulation to the real-world. A dominant strategy for reducing the sim-to-real gap is domain randomization (DR)…
Social Choice Should Guide AI Alignment in Dealing with Diverse Human Feedback
Vincent Conitzer, Rachel Freedman, Jobst Heitzig +9
Foundation models such as GPT-4 are fine-tuned to avoid unsafe or otherwise problematic behavior, such as helping to commit crimes or producing racist text. One approach to fine-tu…
Active teacher selection for reward learning
Rachel Freedman, Justin Svegliato, Kyle Wray +1
Reward learning techniques enable machine learning systems to learn objectives from human feedback. A core limitation of these systems is their assumption that all feedback comes f…
A Generalized Mean Field Algorithm for Variational Inference in Exponential Families
Eric P. Xing, Michael I. Jordan, Stuart Russell
The mean field methods, which entail approximating intractable probability distributions variationally with distributions from a tractable family, enjoy high efficiency, guaranteed…
Learning Plannable Representations with Causal InfoGAN
Thanard Kurutach, Aviv Tamar, Ge Yang +2
In recent years, deep generative models have been shown to 'imagine' convincing high-dimensional observations such as images, audio, and even video, learning directly from raw data…
Learning the Structure of Dynamic Probabilistic Networks
Nir Friedman, Kevin Murphy, Stuart Russell
Dynamic probabilistic networks are a compact representation of complex stochastic processes. In this paper we examine how to learn the structure of a DPN from data. We extend struc…
Adversarial Policies Beat Superhuman Go AIs
Tony T. Wang, Adam Gleave, Tom Tseng +8
We attack the state-of-the-art Go-playing AI system KataGo by training adversarial policies against it, achieving a >97% win rate against KataGo running at superhuman settings. Our…
Rao-Blackwellised Particle Filtering for Dynamic Bayesian Networks
Arnaud Doucet, Nando de Freitas, Kevin Murphy +1
Particle filters (PFs) are powerful sampling-based inference/learning algorithms for dynamic Bayesian networks (DBNs). They allow us to treat, in a principled way, any type of prob…
Understanding Learned Reward Functions
Eric J. Michaud, Adam Gleave, Stuart Russell
In many real-world tasks, it is not possible to procedurally specify an RL agent's reward function. In such cases, a reward function must instead be learned from interacting with a…
Emergence of Superposition: Unveiling the Training Dynamics of Chain of Continuous Thought
Hanlin Zhu, Shibo Hao, Zhiting Hu +3
Previous work shows that the chain of continuous thought (continuous CoT) improves the reasoning capability of large language models (LLMs) by enabling implicit parallel thinking,…
Cooperative and uncooperative institution designs: Surprises and problems in open-source game theory
Andrew Critch, Michael Dennis, Stuart Russell
It is increasingly possible for real-world agents, such as software-based agents or human institutions, to view the internal programming of other such agents that they interact wit…
On Representation Complexity of Model-based and Model-free Reinforcement Learning
Hanlin Zhu, Baihe Huang, Stuart Russell
We study the representation complexity of model-based and model-free reinforcement learning (RL) in the context of circuit complexity. We prove theoretically that there exists a br…
ALMANACS: A Simulatability Benchmark for Language Model Explainability
Edmund Mills, Shiye Su, Stuart Russell +1
How do we measure the efficacy of language model explainability methods? While many explainability methods have been developed, they are typically evaluated on bespoke tasks, preve…
Multi-Principal Assistance Games
Arnaud Fickinger, Simon Zhuang, Dylan Hadfield-Menell +1
Assistance games (also known as cooperative inverse reinforcement learning games) have been proposed as a model for beneficial AI, wherein a robotic agent must act on behalf of a h…
Bridging RL Theory and Practice with the Effective Horizon
Cassidy Laidlaw, Stuart Russell, Anca Dragan
Deep reinforcement learning (RL) works impressively in some environments and fails catastrophically in others. Ideally, RL theory should be able to provide an understanding of why…
Inverse reinforcement learning for video games
Aaron Tucker, Adam Gleave, Stuart Russell
Deep reinforcement learning achieves superhuman performance in a range of video game environments, but requires that a designer manually specify a reward function. It is often easi…
Data-Driven Forecasting of three-Component Seismograms Using Transformer Architectures
Waleed Esmail, Stuart Russell, Jana Klinge +2
Forecasting seismic waveforms beyond observed data remains challenging due to the nonlinear, dispersive, and multi-scale nature of seismic wave propagation. In this work, we introd…
Decayed MCMC Filtering
Bhaskara Marthi, Hanna Pasula, Stuart Russell +1
Filtering---estimating the state of a partially observable Markov process from a sequence of observations---is one of the most widely studied problems in control theory, AI, and co…
Servant of Many Masters: Shifting priorities in Pareto-optimal sequential decision-making
Andrew Critch, Stuart Russell
It is often argued that an agent making decisions on behalf of two or more principals who have different utility functions should adopt a {\em Pareto-optimal} policy, i.e., a polic…
For Learning in Symmetric Teams, Local Optima are Global Nash Equilibria
Scott Emmons, Caspar Oesterheld, Andrew Critch +2
Although it has been known since the 1970s that a globally optimal strategy profile in a common-payoff game is a Nash equilibrium, global optimality is a strict requirement that li…
Cooperative Inverse Reinforcement Learning
Dylan Hadfield-Menell, Anca Dragan, Pieter Abbeel +1
For an autonomous system to be helpful to humans and to pose no unwarranted risks, it needs to align its values with those of the humans in its environment in such a way that its a…
RL, but don't do anything I wouldn't do
Michael K. Cohen, Marcus Hutter, Yoshua Bengio +1
In reinforcement learning, if the agent's reward differs from the designers' true utility, even only rarely, the state distribution resulting from the agent's policy can be very ba…
Swift: Compiled Inference for Probabilistic Programming Languages
Yi Wu, Lei Li, Stuart Russell +1
A probabilistic program defines a probability measure over its semantic structures. One common goal of probabilistic programming languages (PPLs) is to compute posterior probabilit…
Active Reward Learning from Multiple Teachers
Peter Barnett, Rachel Freedman, Justin Svegliato +1
Reward learning algorithms utilize human feedback to infer a reward function, which is then used to train an AI system. This human feedback is often a preference comparison, in whi…
Transformers Provably Learn to Internalize Chain-of-Thought
Yixiao Huang, Hanlin Zhu, Zixuan Wang +4
Chain-of-Thought (CoT) prompting substantially improves the sample efficiency of transformers, reducing the complexity of tasks like parity learning from exponential to polynomial…
Robust and Diverse Multi-Agent Learning via Rational Policy Gradient
Niklas Lauffer, Ameesh Shah, Micah Carroll +3
Adversarial optimization algorithms that explicitly search for flaws in agents' policies have been successfully applied to finding robust and diverse policies in multi-agent settin…
Graph partition strategies for generalized mean field inference
Eric P. Xing, Michael I. Jordan, Stuart Russell
An autonomous variational inference algorithm for arbitrary graphical models requires the ability to optimize variational approximations over the space of model parameters as well…
Avoiding Catastrophe in Online Learning by Asking for Help
Benjamin Plaut, Hanlin Zhu, Stuart Russell
Most learning algorithms with formal regret guarantees assume that all mistakes are recoverable and essentially rely on trying all possible behaviors. This approach is problematic…
Managing extreme AI risks amid rapid progress
Yoshua Bengio, Geoffrey Hinton, Andrew Yao +22
Artificial Intelligence (AI) is progressing rapidly, and companies are shifting their focus to developing generalist AI systems that can autonomously act and pursue goals. Increase…
Who Needs to Know? Minimal Knowledge for Optimal Coordination
Niklas Lauffer, Ameesh Shah, Micah Carroll +2
To optimally coordinate with others in cooperative games, it is often crucial to have information about one's collaborators: successful driving requires understanding which side of…
Provably Optimal Learning Algorithms for Assistance Games
Nivasini Ananthakrishnan, Mark Bedaywi, Michael I. Jordan +2
This paper studies an online variant of the assistance games framework, where an informed agent and an uninformed agent repeatedly interact over timesteps to optimize a common…
Monitoring Latent World States in Language Models with Propositional Probes
Jiahai Feng, Stuart Russell, Jacob Steinhardt
Language models are susceptible to bias, sycophancy, backdoors, and other tendencies that lead to unfaithful responses to the input context. Interpreting internal states of languag…
Cross-Domain Imitation Learning via Optimal Transport
Arnaud Fickinger, Samuel Cohen, Stuart Russell +1
Cross-domain imitation learning studies how to leverage expert demonstrations of one agent to train an imitation agent with a different embodiment or morphology. Comparing trajecto…
Adversarial Policies: Attacking Deep Reinforcement Learning
Adam Gleave, Michael Dennis, Cody Wild +3
Deep reinforcement learning (RL) policies are known to be vulnerable to adversarial perturbations to their observations, similar to adversarial examples for classifiers. However, a…
Efficient Gradient Estimation for Motor Control Learning
Gregory Lawrence, Noah Cowan, Stuart Russell
The task of estimating the gradient of a function in the presence of noise is central to several forms of reinforcement learning, including policy search methods. We present two te…
Evidence of Learned Look-Ahead in a Chess-Playing Neural Network
Erik Jenner, Shreyas Kapur, Vasil Georgiev +3
Do neural networks learn to implement algorithms such as look-ahead or search "in the wild"? Or do they rely purely on collections of simple heuristics? We present evidence of lear…
AssistanceZero: Scalably Solving Assistance Games
Cassidy Laidlaw, Eli Bronstein, Timothy Guo +5
Assistance games are a promising alternative to reinforcement learning from human feedback (RLHF) for training AI assistants. Assistance games resolve key drawbacks of RLHF, such a…
MADE: Exploration via Maximizing Deviation from Explored Regions
Tianjun Zhang, Paria Rashidinejad, Jiantao Jiao +3
In online reinforcement learning (RL), efficient exploration remains particularly challenging in high-dimensional environments with sparse rewards. In low-dimensional environments,…
A compact, hierarchical Q-function decomposition
Bhaskara Marthi, Stuart Russell, David Andre
Previous work in hierarchical reinforcement learning has faced a dilemma: either ignore the values of different possible exit states from a subroutine, thereby risking suboptimal b…
Evidence for a kilometre-scale seismically slow layer atop the core-mantle boundary from normal modes
Stuart Russell, Jessica C. E. Irving, Lisanne Jagt +1
Geodynamic modelling and seismic studies have highlighted the possibility that a thin layer of low seismic velocities, potentially molten, may sit atop the core-mantle boundary but…
Accumulating Risk Capital Through Investing in Cooperation
Charlotte Roman, Michael Dennis, Andrew Critch +1
Recent work on promoting cooperation in multi-agent learning has resulted in many methods which successfully promote cooperation at the cost of becoming more vulnerable to exploita…
Learning and Planning with a Semantic Model
Yi Wu, Yuxin Wu, Aviv Tamar +3
Building deep reinforcement learning agents that can generalize and adapt to unseen environments remains a fundamental challenge for AI. This paper describes progresses on this cha…
Extractive Structures Learned in Pretraining Enable Generalization on Finetuned Facts
Jiahai Feng, Stuart Russell, Jacob Steinhardt
Pretrained language models (LMs) can generalize to implications of facts that they are finetuned on. For example, if finetuned on ``John Doe lives in Tokyo," LMs can correctly answ…
Bayesian Relational Memory for Semantic Visual Navigation
Yi Wu, Yuxin Wu, Aviv Tamar +3
We introduce a new memory architecture, Bayesian Relational Memory (BRM), to improve the generalization ability for semantic visual navigation agents in unseen environments, where…
SLIP: Learning to Predict in Unknown Dynamical Systems with Long-Term Memory
Paria Rashidinejad, Jiantao Jiao, Stuart Russell
We present an efficient and practical (polynomial time) algorithm for online prediction in unknown and partially observed linear dynamical systems (LDS) under stochastic noise. Whe…
BAMDP Shaping: a Unified Framework for Intrinsic Motivation and Reward Shaping
Aly Lidayan, Michael Dennis, Stuart Russell
Intrinsic motivation and reward shaping guide reinforcement learning (RL) agents by adding pseudo-rewards, which can lead to useful emergent behaviors. However, they can also encou…
How Do LLMs Perform Two-Hop Reasoning in Context?
Tianyu Guo, Hanlin Zhu, Ruiqi Zhang +4
``Socrates is human. All humans are mortal. Therefore, Socrates is mortal.'' This form of argument illustrates a typical pattern of two-hop reasoning. Formally, two-hop reasoning r…
Improving Gradient Estimation by Incorporating Sensor Data
Gregory Lawrence, Stuart Russell
An efficient policy search algorithm should estimate the local gradient of the objective function, with respect to the policy parameters, from as few trials as possible. Whereas mo…
Multi-Principal Assistance Games: Definition and Collegial Mechanisms
Arnaud Fickinger, Simon Zhuang, Andrew Critch +2
We introduce the concept of a multi-principal assistance game (MPAG), and circumvent an obstacle in social choice theory, Gibbard's theorem, by using a sufficiently collegial prefe…
The Singapore Consensus on Global AI Safety Research Priorities
Yoshua Bengio, Tegan Maharaj, Luke Ong +84
Rapidly improving AI capabilities and autonomy hold significant promise of transformation, but are also driving vigorous debate on how to ensure that AI is safe, i.e., trustworthy,…
Optimal Conservative Offline RL with General Function Approximation via Augmented Lagrangian
Paria Rashidinejad, Hanlin Zhu, Kunhe Yang +2
Offline reinforcement learning (RL), which refers to decision-making from a previously-collected dataset of interactions, has received significant attention over the past years. Mu…
Estimating and Penalizing Induced Preference Shifts in Recommender Systems
Micah Carroll, Anca Dragan, Stuart Russell +1
The content that a recommender system (RS) shows to users influences them. Therefore, when choosing a recommender to deploy, one is implicitly also choosing to induce specific inte…
Explore and Control with Adversarial Surprise
Arnaud Fickinger, Natasha Jaques, Samyak Parajuli +5
Unsupervised reinforcement learning (RL) studies how to leverage environment statistics to learn useful behaviors without the cost of reward engineering. However, a central challen…
GSM-Agent: Understanding Agentic Reasoning Using Controllable Environments
Hanlin Zhu, Tianyu Guo, Song Mei +4
As LLMs are increasingly deployed as agents, agentic reasoning - the ability to combine tool use, especially search, and reasoning - becomes a critical skill. However, it is hard t…
An Efficient, Generalized Bellman Update For Cooperative Inverse Reinforcement Learning
Dhruv Malik, Malayandi Palaniappan, Jaime F. Fisac +3
Our goal is for AI systems to correctly identify and act according to their human user's objectives. Cooperative Inverse Reinforcement Learning (CIRL) formalizes this value alignme…
International AI Safety Report 2026
Yoshua Bengio, Stephen Clare, Carina Prunkl +89
The International AI Safety Report 2026 synthesises the current scientific evidence on the capabilities, emerging risks, and safety of general-purpose AI systems. The report series…
Probabilistic Model-Based Approach for Heart Beat Detection
Hugh Chen, Yusuf Erol, Eric Shen +1
Nowadays, hospitals are ubiquitous and integral to modern society. Patients flow in and out of a veritable whirlwind of paperwork, consultations, and potential inpatient admissions…
Observation Interference in Partially Observable Assistance Games
Scott Emmons, Caspar Oesterheld, Vincent Conitzer +1
We study partially observable assistance games (POAGs), a model of the human-AI value alignment problem which allows the human and the AI assistant to have partial observations. Mo…
Gibbs Sampling in Open-Universe Stochastic Languages
Nimar S. Arora, Rodrigo de Salvo Braz, Erik B. Sudderth +1
Languages for open-universe probabilistic models (OUPMs) can represent situations with an unknown number of objects and iden- tity uncertainty. While such cases arise in a wide ran…
Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought
Hanlin Zhu, Shibo Hao, Zhiting Hu +3
Large Language Models (LLMs) have demonstrated remarkable performance in many applications, including challenging reasoning problems via chain-of-thoughts (CoTs) techniques that ge…
Discrete-Continuous Mixtures in Probabilistic Programming: Generalized Semantics and Inference Algorithms
Yi Wu, Siddharth Srivastava, Nicholas Hay +2
Despite the recent successes of probabilistic programming languages (PPLs) in AI applications, PPLs offer only limited support for random variables whose distributions combine disc…
Forecasting Seismic Waveforms: A Deep Learning Approach for Einstein Telescope
Waleed Esmail, Alexander Kappes, Stuart Russell +1
We introduce \textit{SeismoGPT}, a transformer-based model for forecasting three-component seismic waveforms in the context of future gravitational wave detectors like the Einstein…
Stochastic Simulation Algorithms for Dynamic Probabilistic Networks
Keiji Kanazawa, Daphne Koller, Stuart Russell
Stochastic simulation algorithms such as likelihood weighting often give fast, accurate approximations to posterior probabilities in probabilistic networks, and are the methods of…
The Partially Observable Off-Switch Game
Andrew Garber, Rohan Subramani, Linus Luu +3
A wide variety of goals could cause an AI to disable its off switch because "you can't fetch the coffee if you're dead" (Russell 2019). Prior theoretical work on this shutdown prob…
The MineRL BASALT Competition on Learning from Human Feedback
Rohin Shah, Cody Wild, Steven H. Wang +10
The last decade has seen a significant increase of interest in deep learning research, with many public successes that have demonstrated its potential. As such, these systems are n…
Scalable Online Planning via Reinforcement Learning Fine-Tuning
Arnaud Fickinger, Hengyuan Hu, Brandon Amos +2
Lookahead search has been a critical component of recent AI successes, such as in the games of chess, go, and poker. However, the search methods used in these games, and in many ot…
Pruned Neural Networks are Surprisingly Modular
Daniel Filan, Shlomi Hod, Cody Wild +2
The learned weights of a neural network are often considered devoid of scrutable internal structure. To discern structure in these weights, we introduce a measurable notion of modu…
International AI Safety Report
Yoshua Bengio, Sören Mindermann, Daniel Privitera +93
The first International AI Safety Report comprehensively synthesizes the current evidence on the capabilities, risks, and safety of advanced AI systems. The report was mandated by…
Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game
Sam Toyer, Olivia Watkins, Ethan Adrian Mendes +9
While Large Language Models (LLMs) are increasingly being used in real-world applications, they remain vulnerable to prompt injection attacks: malicious third party prompts that su…
Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
David "davidad" Dalrymple, Joar Skalse, Yoshua Bengio +14
Ensuring that AI systems reliably and robustly avoid harmful or dangerous behaviours is a crucial challenge, especially for AI systems with a high degree of autonomy and general in…
Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics
Hanlin Zhu, Baihe Huang, Shaolun Zhang +4
Auto-regressive large language models (LLMs) show impressive capacities to solve many complex reasoning tasks while struggling with some simple logical reasoning tasks such as inve…
DiscoLoop: Looping Discrete Embeddings and Continuous Hidden States for Multi-hop Reasoning
Hengyu Fu, Tianyu Guo, Zixuan Wang +5
Large language models achieve strong performance on many reasoning tasks when allowed to externalize intermediate steps as Chain-of-Thought (CoT). However, many questions require t…
imitation: Clean Imitation Learning Implementations
Adam Gleave, Mohammad Taufeeque, Juan Rocamonde +7
imitation provides open-source implementations of imitation and reward learning algorithms in PyTorch. We include three inverse reinforcement learning (IRL) algorithms, three imita…
Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Design
Michael Dennis, Natasha Jaques, Eugene Vinitsky +4
A wide range of reinforcement learning (RL) problems - including robustness, transfer learning, unsupervised RL, and emergent complexity - require specifying a distribution of task…
AI Alignment with Changing and Influenceable Reward Functions
Micah Carroll, Davis Foote, Anand Siththaranjan +2
Existing AI alignment approaches assume that preferences are static, which is unrealistic: our preferences change, and may even be influenced by our interactions with AI systems th…
An Empirical Investigation of Representation Learning for Imitation
Xin Chen, Sam Toyer, Cody Wild +9
Imitation learning often needs a large demonstration set in order to handle the full range of situations that an agent might find itself in during deployment. However, collecting e…
When Do Autoregressive Sequence Models Forecast Physical Wavefields? A Controlled Study on Synthetic Seismograms
Waleed Esmail, Stuart Russell, Jana Klinge +2
Long-horizon autoregressive forecasting of oscillatory physical signals, such as seismograms, gravitational-wave strain, and similar wavefields is limited by error accumulation: as…
A Re-examination of Ellipticity Corrections for Seismic Phases
Stuart Russell, John F. Rudge, Jessica C. E. Irving +1
The Earth's ellipticity of figure has an effect on the travel times of seismic waves over teleseismic distances. Tables of ellipticity corrections and coefficients have been used b…
Uncertain Decisions Facilitate Better Preference Learning
Cassidy Laidlaw, Stuart Russell
Existing observational approaches for learning human preferences, such as inverse reinforcement learning, usually make strong assumptions about the observability of the human's env…
Selecting Computations: Theory and Applications
Nicholas Hay, Stuart Russell, David Tolpin +1
Sequential decision problems are often approximately solvable by simulating possible future action sequences. Metalevel decision procedures have been developed for selecting which…
The MAGICAL Benchmark for Robust Imitation
Sam Toyer, Rohin Shah, Andrew Critch +1
Imitation Learning (IL) algorithms are typically evaluated in the same environment that was used to create demonstrations. This rewards precise reproduction of demonstrations in on…
Research Priorities for Robust and Beneficial Artificial Intelligence
Stuart Russell, Daniel Dewey, Max Tegmark
Success in the quest for artificial intelligence has the potential to bring unprecedented benefits to humanity, and it is therefore worthwhile to investigate how to maximize these…
Variational MCMC
Nando de Freitas, Pedro Hojen-Sorensen, Michael I. Jordan +1
We propose a new class of learning algorithms that combines variational approximation and Markov chain Monte Carlo (MCMC) simulation. Naive algorithms that use the variational appr…
International Scientific Report on the Safety of Advanced AI (Interim Report)
Yoshua Bengio, Sören Mindermann, Daniel Privitera +41
This is the interim publication of the first International Scientific Report on the Safety of Advanced AI. The report synthesises the scientific understanding of general-purpose AI…
Safe Learning Under Irreversible Dynamics via Asking for Help
Benjamin Plaut, Juan Liévano-Karim, Hanlin Zhu +1
Most learning algorithms with formal regret guarantees essentially rely on trying all possible behaviors, which is problematic when some errors cannot be recovered from. Instead, w…
Should Robots be Obedient?
Smitha Milli, Dylan Hadfield-Menell, Anca Dragan +1
Intuitively, obedience -- following the order that a human gives -- seems like a good property for a robot to have. But, we humans are not perfect and we may give orders that are n…