An Introduction to Collective Intelligence
arXiv:cs/9908014
Abstract
This paper surveys the emerging science of how to design a ``COllective INtelligence'' (COIN). A COIN is a large multi-agent system where: (i) There is little to no centralized communication or control; and (ii) There is a provided world utility function that rates the possible histories of the full system. In particular, we are interested in COINs in which each agent runs a reinforcement learning (RL) algorithm. Rather than use a conventional modeling approach (e.g., model the system dynamics, and hand-tune agents to cooperate), we aim to solve the COIN design problem implicitly, via the ``adaptive'' character of the RL algorithms of each of the agents. This approach introduces an entirely new, profound design problem: Assuming the RL algorithms are able to achieve high rewards, what reward functions for the individual agents will, when pursued by those agents, result in high world utility? In other words, what reward functions will best ensure that we do not have phenomena like the tragedy of the commons, Braess's paradox, or the liquidity trap? Although still very young, research specifically concentrating on the COIN design problem has already resulted in successes in artificial domains, in particular in packet-routing, the leader-follower problem, and in variants of Arthur's El Farol bar problem. It is expected that as it matures and draws upon other disciplines related to COINs, this research will greatly expand the range of tasks addressable by human engineers. Moreover, in addition to drawing on them, such a fully developed scie nce of COIN design may provide much insight into other already established scientific fields, such as economics, game theory, and population biology.
88 pages, 10 figs, 297 refs
References in corpus (22)
- Evolutionary prisoner's dilemma game on a square lattice
- Self Organization and a Dynamical Transition in Traffic Flow Models
- Discrete stochastic models for traffic flow
- Decoherence, Chaos, and the Second Law
- Active Walker Model for the Formation of Human and Animal Trail Systems
- Phase Diagram of Traffic States in the Presence of Inhomogeneities
- Irrelevance of memory in the minority game
- Using Collective Intelligence to Route Internet Traffic
- Anytime Coalition Structure Generation with Worst Case Guarantees
- Modeling Market Mechanism with Evolutionary Games
- Matrix Games, Mixed Strategies, and Statistical Mechanics
- The Dynamics of Money
- Autocatalytic closure in a cognitive system: A tentative scenario for the origin of culture
- Statistical mechanics of voting
- Competition, efficiency and collective behavior in the "El Farol" bar model
- Learning Conventions in Multiagent Stochastic Domains using Likelihood Estimates
- Adaptive Competition, Market Efficiency, Phase Transitions and Spin-Glasses
- A simple model for the formation of a complex organism
- Avoiding Braess' Paradox through Collective Intelligence
- Spontaneous coalition forming: a model from spin glass
- Distributed Self-regulation Induced by Negative Feedbacks in Ecological and Economic Systems
- Coherence and clustering in ensembles of neural networks
Cited by in corpus (10)
- Embodied Evolution in Collective Robotics: A Review
- Collective Intelligence for Control of Distributed Dynamical Systems
- Collective Intelligence, Data Routing and Braess' Paradox
- Learning Reciprocity in Complex Sequential Social Dilemmas
- Difference Rewards Policy Gradients
- A Method for Solving Distributed Service Allocation Problems
- Avoiding Braess' Paradox through Collective Intelligence
- Distributed Adaptive Reinforcement Learning: A Method for Optimal Routing
- Emergence is coupled to scope, not level
- Evolutionary Optimization in an Algorithmic Setting