activity
20172024
most citedIncentives for Responsiveness, Instrumental Control and Impact

13 citations · 34 across the 9 of their papers we have counts for

collaborators

11 papers

cs.LG2024★ 1 cited

DE-HNN: An effective neural model for Circuit Netlist representation

Zhishang Luo, Truong Son Hy, Puoya Tabaghi +7

The run-time for optimization tools used in chip design has grown with the complexity of designs to the point where it can take several days to go through one design cycle which ha…

cs.AI2023★ 1 cited

Human Control: Definitions and Algorithms

Ryan Carey, Tom Everitt

How can humans stay in control of advanced artificial intelligence systems? One proposal is corrigibility, which requires the agent to follow the instructions of a human overseer,…

cs.AI2023★ 12 cited

Reasoning about Causality in Games

Lewis Hammond, James Fox, Tom Everitt +3

Causal reasoning and game-theoretic reasoning are fundamental topics in artificial intelligence, among many other disciplines: this paper is concerned with their intersection. Desp…

cs.AI2022★ 3 cited

Path-Specific Objectives for Safer Agent Incentives

Sebastian Farquhar, Ryan Carey, Tom Everitt

We present a general framework for training safe agents whose naive incentives are unsafe. As an example, manipulative or deceptive behaviour can improve rewards but should be avoi…

cs.LG2022★ 1 cited

Too Big to Fail? Active Few-Shot Learning Guided Logic Synthesis

Animesh Basak Chowdhury, Benjamin Tan, Ryan Carey +3

Generating sub-optimal synthesis transformation sequences ("synthesis recipe") is an important problem in logic synthesis. Manually crafted synthesis recipes have poor quality. Sta…

cs.AI2022

A Complete Criterion for Value of Information in Soluble Influence Diagrams

Chris van Merwijk, Ryan Carey, Tom Everitt

Influence diagrams have recently been used to analyse the safety and fairness properties of AI systems. A key building block for this analysis is a graphical criterion for value of…

cs.LG2022★ 3 cited

Why Fair Labels Can Yield Unfair Predictions: Graphical Conditions for Introduced Unfairness

Carolyn Ashurst, Ryan Carey, Silvia Chiappa +1

In addition to reproducing discriminatory relationships in the training data, machine learning systems can also introduce or amplify discriminatory effects. We refer to this as int…

cs.AI2021

Agent Incentives: A Causal Perspective

Tom Everitt, Ryan Carey, Eric Langlois +2

We present a framework for analysing agent incentives using causal influence diagrams. We establish that a well-known criterion for value of information is complete. We propose a n…

cs.AI2020★ 13 cited

Incentives for Responsiveness, Instrumental Control and Impact

Ryan Carey, Eric Langlois, Chris van Merwijk +2

We introduce three concepts that describe an agent's incentives: response incentives indicate which variables in the environment, such as sensitive demographic information, affect…

cs.AI2019

(When) Is Truth-telling Favored in AI Debate?

Vojtěch Kovařík, Ryan Carey

For some problems, humans may not be able to accurately judge the goodness of AI-proposed solutions. Irving et al. (2018) propose that in such cases, we may use a debate between tw…

cs.AI2017

Incorrigibility in the CIRL Framework

Ryan Carey

A value learning system has incentives to follow shutdown instructions, assuming the shutdown instruction provides information (in the technical sense) about which actions lead to…