Parametric-Task MAP-Elites

Abstract

Optimizing a set of functions simultaneously by leveraging their similarity is called multi-task optimization. Current black-box multi-task algorithms only solve a finite set of tasks, even when the tasks originate from a continuous space. In this paper, we introduce Parametric-Task MAP-Elites (PT-ME), a new black-box algorithm for continuous multi-task optimization problems. This algorithm (1) solves a new task at each iteration, effectively covering the continuous space, and (2) exploits a new variation operator based on local linear regression. The resulting dataset of solutions makes it possible to create a function that maps any task parameter to its optimal solution. We show that PT-ME outperforms all baselines, including the deep reinforcement learning algorithm PPO on two parametric-task toy problems and a robotic problem in simulation.

References in corpus (11)

• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• A Brief Survey of Deep Reinforcement Learning
• An Introduction to Deep Reinforcement Learning
• Robots that can adapt like animals
• Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots
• Illuminating search spaces by mapping elites
• Centralized Cooperation for Connected and Automated Vehicles at Intersections by Proximal Policy Optimization
• Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space
• Quality Diversity for Multi-task Optimization
• Strategies for Using Proximal Policy Optimization in Mobile Puzzle Games
• Multi-Task Multi-Behavior MAP-Elites