Optimal foraging strategies can be learned

Abstract

The foraging behavior of animals is a paradigm of target search in nature. Understanding which foraging strategies are optimal and how animals learn them are central challenges in modeling animal foraging. While the question of optimality has wide-ranging implications across fields such as economy, physics, and ecology, the question of learnability is a topic of ongoing debate in evolutionary biology. Recognizing the interconnected nature of these challenges, this work addresses them simultaneously by exploring optimal foraging strategies through a reinforcement learning framework. To this end, we model foragers as learning agents. We first prove theoretically that maximizing rewards in our reinforcement learning model is equivalent to optimizing foraging efficiency. We then show with numerical experiments that, in the paradigmatic model of non-destructive search, our agents learn foraging strategies which outperform the efficiency of some of the best known strategies such as Lévy walks. These findings highlight the potential of reinforcement learning as a versatile framework not only for optimizing search strategies but also to model the learning process, thus shedding light on the role of learning in natural optimization processes.

arXiv comment12 pages (6 main text) and 9 figures. Codes and tutorials available at: https://gorkamunoz.github.io/rl_opts/

References in corpus (17)

• Power-law distributions in empirical data
• Lévy walks
• Tune: A Research Platform for Distributed Model Selection and Training
• The topography of the environment alters the optimal search strategy for active particles
• Optimizing persistent random searches
• Inverse square Lévy walks are not optimal search strategies for $d\ge 2$
• Optimal search strategies of run-and-tumble walks
• Projective simulation for classical learning agents: a comprehensive investigation
• Projective simulation with generalization
• Optimal non-Markovian search strategies with n-step memory
• Meta-learning within Projective Simulation
• Phase transition in non-Markovian animal exploration model with preferential returns
• Collective defense of honeybee colonies: experimental results and theoretical modeling
• Comparison of pure and combined search strategies for single and multiple targets
• Development of swarm behavior in artificial learning agents that adapt to different foraging environments
• On the convergence of projective-simulation-based reinforcement learning in Markov decision processes
• Adaptive active Brownian particles searching for targets of unknown positions

Cited by in corpus (5)

• Efficient Rare Event Sampling with Unsupervised Normalising Flows
• Learning how to find targets in the micro-world: The case of intermittent active Brownian particles
• Reinforcement Learning for Active Matter
• Run-and-Tumble Particles Learning Chemotaxis
• Multi-Excitation Projective Simulation with a Many-Body Physics Inspired Inductive Bias