AI Research Considerations for Human Existential Safety (ARCHES)
arXiv:2006.04948
Abstract
Framed in positive terms, this report examines how technical AI research might be steered in a manner that is more attentive to humanity's long-term prospects for survival as a species. In negative terms, we ask what existential risks humanity might face from AI development in the next century, and by what principles contemporary technical research might be directed to address those risks. A key property of hypothetical AI technologies is introduced, called \emph{prepotence}, which is useful for delineating a variety of potential existential risks from artificial intelligence, even as AI paradigms might shift. A set of \auxref{dirtot} contemporary research \directions are then examined for their potential benefit to existential safety. Each research direction is explained with a scenario-driven motivation, and examples of existing work from which to build. The research directions present their own risks and benefits to society that could occur at various scales of impact, and in particular are not guaranteed to benefit existential safety if major developments in them are deployed without adequate forethought and oversight. As such, each direction is accompanied by a consideration of potentially negative side effects.
References in corpus (13)
- Explaining and Harnessing Adversarial Examples
- On Calibration of Modern Neural Networks
- Understanding Neural Networks Through Deep Visualization
- Multi-agent Reinforcement Learning in Sequential Social Dilemmas
- An approach to reachability analysis for feed-forward ReLU neural networks
- AI Safety Gridworlds
- Trial without Error: Towards Safe Reinforcement Learning via Human Intervention
- Developing Bug-Free Machine Learning Systems With Formal Mathematics
- Agent-Agnostic Human-in-the-Loop Reinforcement Learning
- Federated Control with Hierarchical Multi-Agent Deep Reinforcement Learning
- Literal or Pedagogic Human? Analyzing Human Model Misspecification in Objective Learning
- Scaling shared model governance via model splitting
- Cause, Responsibility, and Blame: oA Structural-Model Approach