Bridging adaptive management and reinforcement learning for more robust decisions

Abstract

From out-competing grandmasters in chess to informing high-stakes healthcare decisions, emerging methods from artificial intelligence are increasingly capable of making complex and strategic decisions in diverse, high-dimensional, and uncertain situations. But can these methods help us devise robust strategies for managing environmental systems under great uncertainty? Here we explore how reinforcement learning, a subfield of artificial intelligence, approaches decision problems through a lens similar to adaptive environmental management: learning through experience to gradually improve decisions with updated knowledge. We review where reinforcement learning (RL) holds promise for improving evidence-informed adaptive management decisions even when classical optimization methods are intractable. For example, model-free deep RL might help identify quantitative decision strategies even when models are nonidentifiable. Finally, we discuss technical and social issues that arise when applying reinforcement learning to adaptive management problems in the environmental domain. Our synthesis suggests that environmental management and computer science can learn from one another about the practices, promises, and perils of experience-based decision-making.

In press at Philosophical Transactions of the Royal Society B

References in corpus (12)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence
• Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
• Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey
• Uncertainty-Aware Reinforcement Learning for Collision Avoidance
• Offline Reinforcement Learning with Implicit Q-Learning
• Learning to Utilize Shaping Rewards: A New Approach of Reward Shaping
• Multi-Objective Deep Reinforcement Learning
• Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Traffic
• The Forecast Trap
• Offline Reinforcement Learning with Instrumental Variables in Confounded Markov Decision Processes
• Reward Reports for Reinforcement Learning

Cited by in corpus (1)

• Pretty darn good control: when are approximate solutions better than approximate models