On Controllability of AI
arXiv:2008.04071
Abstract
Invention of artificial general intelligence is predicted to cause a shift in the trajectory of human civilization. In order to reap the benefits and avoid pitfalls of such powerful technology it is important to be able to control it. However, possibility of controlling artificial general intelligence and its more advanced version, superintelligence, has not been formally established. In this paper, we present arguments as well as supporting evidence from multiple domains indicating that advanced AI can't be fully controlled. Consequences of uncontrollability of AI are discussed with respect to future of humanity and research on AI, and AI safety and security.
References in corpus (29)
- Language Models are Few-Shot Learners
- On the (im)possibility of fairness
- Information-theoretic approach to the study of control systems
- Information-Theoretic Limits of Control
- Scalable agent alignment via reward modeling: a research direction
- Towards Verified Artificial Intelligence
- Physical limits of inference
- Superintelligence cannot be contained: Lessons from Computability Theory
- Towards Moral Autonomous Systems
- Unethical Research: How to Create a Malevolent Artificial Intelligence
- Occam's razor is insufficient to infer the preferences of irrational agents
- AI Research Considerations for Human Existential Safety (ARCHES)
- From 'F' to 'A' on the N.Y. Regents Science Exams: An Overview of the Aristo Project
- Risks from Learned Optimization in Advanced Machine Learning Systems
- The impossibility of "fairness": a generalized impossibility result for decisions
- Impossibility and Uncertainty Theorems in AI Value Alignment (or why your AGI should not have a utility function)
- Building Safer AGI by introducing Artificial Stupidity
- Towards Resolving Unidentifiability in Inverse Reinforcement Learning
- Personal Universes: A Solution to the Multi-Agent Value Alignment Problem
- Unexplainability and Incomprehensibility of Artificial Intelligence
- Construction of an NP Problem with an Exponential Lower Bound
- Requisite Variety in Ethical Utility Functions for AI Value Alignment
- Stovepiping and Malicious Software: A Critical Review of AGI Containment
- Emergence of Addictive Behaviors in Reinforcement Learning Agents
- Interstellar communication. IX. Message decontamination is impossible
- A Cyber Science Based Ontology for Artificial General Intelligence Containment
- An AGI Modifying Its Utility Function in Violation of the Orthogonality Thesis
- Constraints on physical reality arising from a formalization of knowledge
- Non-Evolutionary Superintelligences Do Nothing, Eventually