24 citations · 84 across the 14 of their papers we have counts for
16 papers
Temporal Inference with Finite Factored Sets
Scott Garrabrant
We propose a new approach to temporal inference, inspired by the Pearlian causal inference paradigm - though quite different from Pearl's approach formally. Rather than using direc…
Cartesian Frames
Scott Garrabrant, Daniel A. Herrmann, Josiah Lopez-Wild
We introduce a novel framework, the theory of Cartesian frames (CF), that gives powerful tools for manipulating sets of acts. The CF framework takes as its most fundamental buildin…
Risks from Learned Optimization in Advanced Machine Learning Systems
Evan Hubinger, Chris van Merwijk, Vladimir Mikulik +2
We analyze the type of learned optimization that occurs when a learned model (such as a neural network) is itself an optimizer - a situation we refer to as mesa-optimization, a neo…
Embedded Agency
Abram Demski, Scott Garrabrant
Traditional models of rational action treat the agent as though it is cleanly separated from its environment, and can act on that environment from the outside. Such agents have a k…
Categorizing Variants of Goodhart's Law
David Manheim, Scott Garrabrant
There are several distinct failure modes for overoptimization of systems on the basis of metrics. This occurs when a metric which can be used to improve a system is used to an exte…
A Formal Approach to the Problem of Logical Non-Omniscience
Scott Garrabrant, Tsvi Benson-Tilsen, Andrew Critch +2
We present the logical induction criterion for computable algorithms that assign probabilities to every logical statement in a given formal language, and refine those probabilities…