Hierarchical Soft Actor-Critic: Adversarial Exploration via Mutual Information Optimization

Abstract

We describe a novel extension of soft actor-critics for hierarchical Deep Q-Networks (HDQN) architectures using mutual information metric. The proposed extension provides a suitable framework for encouraging explorations in such hierarchical networks. A natural utilization of this framework is an adversarial setting, where meta-controller and controller play minimax over the mutual information objective but cooperate on maximizing expected rewards.

Presented at the ICML 2019 workshop on Imitation, Intent, and Interaction, Long Beach, CA, USA