Differentiable Quality Diversity

Abstract

Quality diversity (QD) is a growing branch of stochastic optimization research that studies the problem of generating an archive of solutions that maximize a given objective function but are also diverse with respect to a set of specified measure functions. However, even when these functions are differentiable, QD algorithms treat them as "black boxes", ignoring gradient information. We present the differentiable quality diversity (DQD) problem, a special case of QD, where both the objective and measure functions are first order differentiable. We then present MAP-Elites via a Gradient Arborescence (MEGA), a DQD algorithm that leverages gradient information to efficiently explore the joint range of the objective and measure functions. Results in two QD benchmark domains and in searching the latent space of a StyleGAN show that MEGA significantly outperforms state-of-the-art QD algorithms, highlighting DQD's promise for efficient quality diversity optimization when gradient information is available. Source code is available at https://github.com/icaros-usc/dqd.

arXiv commentAccepted to NeurIPS 2021 (oral presentation)

References in corpus (10)

• Robots that can adapt like animals
• Illuminating search spaces by mapping elites
• Procedural Content Generation through Quality Diversity
• Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space
• CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoders
• Discovering the Elite Hypervolume by Leveraging Interspecies Correlation
• Effective Diversity in Population Based Reinforcement Learning
• Policy Manifold Search: Exploring the Manifold Hypothesis for Diversity-based Neuroevolution
• CPPN2GAN: Combining Compositional Pattern Producing Networks and GANs for Large-scale Pattern Generation
• BOP-Elites, a Bayesian Optimisation algorithm for Quality-Diversity search

Cited by in corpus (11)

• Language Model Crossover: Variation through Few-Shot Prompting
• Multi-Objective Quality Diversity Optimization
• COIL: Constrained Optimization in Learned Latent Space: Learning Representations for Valid Solutions
• Workspace Optimization Techniques to Improve Prediction of Human Motion During Human-Robot Collaboration
• Don't Bet on Luck Alone: Enhancing Behavioral Reproducibility of Quality-Diversity Solutions in Uncertain Domains
• On the Importance of Environments in Human-Robot Coordination
• Multi-Objective Quality-Diversity for Crystal Structure Prediction
• Relevance-guided Unsupervised Discovery of Abilities with Quality-Diversity Algorithms
• Leveraging Human Feedback to Evolve and Discover Novel Emergent Behaviors in Robot Swarms
• Gradient-Informed Quality Diversity for the Illumination of Discrete Spaces
• Improving the Data Efficiency of Multi-Objective Quality-Diversity through Gradient Assistance and Crowding Exploration