24 citations · 84 across the 14 of their papers we have counts for
6 papers · 1 filter
Temporal Inference with Finite Factored Sets
Scott Garrabrant
We propose a new approach to temporal inference, inspired by the Pearlian causal inference paradigm - though quite different from Pearl's approach formally. Rather than using direc…
Risks from Learned Optimization in Advanced Machine Learning Systems
Evan Hubinger, Chris van Merwijk, Vladimir Mikulik +2
We analyze the type of learned optimization that occurs when a learned model (such as a neural network) is itself an optimizer - a situation we refer to as mesa-optimization, a neo…
Embedded Agency
Abram Demski, Scott Garrabrant
Traditional models of rational action treat the agent as though it is cleanly separated from its environment, and can act on that environment from the outside. Such agents have a k…
Categorizing Variants of Goodhart's Law
David Manheim, Scott Garrabrant
There are several distinct failure modes for overoptimization of systems on the basis of metrics. This occurs when a metric which can be used to improve a system is used to an exte…
Logical Induction
Scott Garrabrant, Tsvi Benson-Tilsen, Andrew Critch +2
We present a computable algorithm that assigns probabilities to every logical statement in a given formal language, and refines those probabilities over time. For instance, if the…
Inductive Coherence
Scott Garrabrant, Benya Fallenstein, Abram Demski +1
While probability theory is normally applied to external environments, there has been some recent interest in probabilistic modeling of the outputs of computations that are too exp…