Deep reinforcement learning for the olfactory search POMDP: a quantitative benchmark

Abstract

The olfactory search POMDP (partially observable Markov decision process) is a sequential decision-making problem designed to mimic the task faced by insects searching for a source of odor in turbulence, and its solutions have applications to sniffer robots. As exact solutions are out of reach, the challenge consists in finding the best possible approximate solutions while keeping the computational cost reasonable. We provide a quantitative benchmarking of a solver based on deep reinforcement learning against traditional POMDP approximate solvers. We show that deep reinforcement learning is a competitive alternative to standard methods, in particular to generate lightweight policies suitable for robots.

References in corpus (5)

• Anytime Point-Based Approximations for Large POMDPs
• Perseus: Randomized Point-based Value Iteration for POMDPs
• Partially Observable Markov Decision Processes (POMDPs) and Robotics
• Searching for a source without gradients: how good is infotaxis and how to beat it
• Optimal policies for Bayesian olfactory search in turbulent flows

Cited by in corpus (5)

• Optimal policies for Bayesian olfactory search in turbulent flows
• A critical assessment of reinforcement learning methods for microswimmer navigation in complex flows
• Optimal trajectories for Bayesian olfactory search in turbulent flows: the low information limit and beyond
• Policy heterogeneity improves collective olfactory search in 3-D turbulence
• Multi-source Plume Tracing via Multi-Agent Reinforcement Learning