Unstable Dynamics of Adaptation in Unknown Environment due to Novelty Seeking
arXiv:1305.3657 · doi:10.1142/S0219525914500131
Abstract
Learning and adaptation play great role in emergent socio-economic phenomena. Complex dynamics has been previously found in the systems of multiple learning agents interacting via a simple game. Meanwhile, the single agent adaptation is considered trivially stable. We advocate the idea that adopting a more complex model of the individual behavior may result in a more diverse spectrum of macro-level behaviors. We develop an adaptation model based on the reinforcement learning framework extended by an additional processing channel. We scrutiny the dynamics of the single agent adapting to the unknown environment; the agent is biased by novelty seeking, the intrinsic inclination for exploration. We demonstrate that the behavior of the novelty-seeking agent may be inherently unstable. One of the surprising results is that under certain conditions the increase of the novelty-seeking level may cause the agent to switch from the non-rational to the strictly rational behavior. Our results give evidence to the hypothesis that the intrinsic motives of agents should be paid no less attention than the extrinsic ones in the models of complex socio-economic systems.
arXiv comment18 pages, 6 figures
References in corpus (7)
- Coupled Replicator Equations for the Dynamics of Learning in Multiagent Systems
- Dynamics of Boltzmann Q-Learning in Two-Player Two-Action Games
- Stability and Diversity in Collective Adaptation
- Intrinsic noise in game dynamical learning
- Cycles of cooperation and defection in imperfect learning
- Extended phase space description of human-controlled systems dynamics
- Scale-free memory model for multiagent reinforcement learning. Mean field approximation and rock-paper-scissors dynamics