Open Problems in Cooperative AI
arXiv:2012.08630
Abstract
Problems of cooperation--in which agents seek ways to jointly improve their welfare--are ubiquitous and important. They can be found at scales ranging from our daily routines--such as driving on highways, scheduling meetings, and working collaboratively--to our global challenges--such as peace, commerce, and pandemic preparedness. Arguably, the success of the human species is rooted in our ability to cooperate. Since machines powered by artificial intelligence are playing an ever greater role in our lives, it will be important to equip them with the capabilities necessary to cooperate and to foster cooperation. We see an opportunity for the field of artificial intelligence to explicitly focus effort on this class of problems, which we term Cooperative AI. The objective of this research would be to study the many aspects of the problems of cooperation and to innovate in AI to contribute to solving these problems. Central goals include building machine agents with the capabilities needed for cooperation, building tools to foster cooperation in populations of (machine and/or human) agents, and otherwise conducting AI research for insight relevant to problems of cooperation. This research integrates ongoing work on multi-agent systems, game theory and social choice, human-machine interaction and alignment, natural-language processing, and the construction of social tools and platforms. However, Cooperative AI is not the union of these existing areas, but rather an independent bet about the productivity of specific kinds of conversations that involve these and other areas. We see opportunity to more explicitly focus on the problem of cooperation, to construct unified theory and vocabulary, and to build bridges with adjacent communities working on cooperation, including in the natural, social, and behavioural sciences.
References in corpus (14)
- Natural Language Processing (almost) from Scratch
- Language Models are Few-Shot Learners
- Equality of Opportunity in Supervised Learning
- Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
- The Unreasonable Effectiveness of Deep Learning in Artificial Intelligence
- On the Utility of Learning about Humans for Human-AI Coordination
- Autocurricula and the Emergence of Innovation from Social Interaction: A Manifesto for Multi-Agent Intelligence Research
- The AI Economist: Improving Equality and Productivity with AI-Driven Tax Policies
- Towards Playing Full MOBA Games with Deep Reinforcement Learning
- Observational Learning by Reinforcement Learning
- AI Research Considerations for Human Existential Safety (ARCHES)
- Delay-Aware Multi-Agent Reinforcement Learning for Cooperative and Competitive Environments
- Emergent Reciprocity and Team Formation from Randomized Uncertain Social Preferences
- Artificial Intelligence & Cooperation