Shaking the foundations: delusions in sequence models for interaction and control

Abstract

The recent phenomenal success of language models has reinvigorated machine learning research, and large sequence models such as transformers are being applied to a variety of domains. One important problem class that has remained relatively elusive however is purposeful adaptive behavior. Currently there is a common perception that sequence models "lack the understanding of the cause and effect of their actions" leading them to draw incorrect inferences due to auto-suggestive delusions. In this report we explain where this mismatch originates, and show that it can be resolved by treating actions as causal interventions. Finally, we show that in supervised learning, one can teach a system to condition or intervene on data by training with factual and counterfactual error signals respectively.

DeepMind Tech Report, 16 pages, 4 figures

References in corpus (9)

• Language Models are Few-Shot Learners
• On the Opportunities and Risks of Foundation Models
• RL${}^2$: Fast Reinforcement Learning via Slow Reinforcement Learning
• Decision Transformer: Reinforcement Learning via Sequence Modeling
• Learning to reinforcement learn
• Invariant Risk Minimization
• Task-Relevant Adversarial Imitation Learning
• Causally Correct Partial Models for Reinforcement Learning
• Causal Analysis of Agent Behavior for AI Safety

Cited by in corpus (2)

• A social path to human-like artificial intelligence
• Emergent Causality and the Foundation of Consciousness