Variational Deep Q Network

Abstract

We propose a framework that directly tackles the probability distribution of the value function parameters in Deep Q Network (DQN), with powerful variational inference subroutines to approximate the posterior of the parameters. We will establish the equivalence between our proposed surrogate objective and variational inference loss. Our new algorithm achieves efficient exploration and performs well on large scale chain Markov Decision Process (MDP).

12 pages, 5 figures, Second workshop on Bayesian Deep Learning (NIPS 2017)

References in corpus (7)

• Dueling Network Architectures for Deep Reinforcement Learning
• Benchmarking Deep Reinforcement Learning for Continuous Control
• Noisy Networks for Exploration
• VIME: Variational Information Maximizing Exploration
• Deep Exploration via Bootstrapped DQN
• Deep Probabilistic Programming
• Bootstrapped Thompson Sampling and Deep Exploration

Cited by in corpus (5)

• Uncertainty Weighted Actor-Critic for Offline Reinforcement Learning
• The Potential of the Return Distribution for Exploration in RL
• Reinforced Few-Shot Acquisition Function Learning for Bayesian Optimization
• Amortized Variational Deep Q Network
• Interpretable Option Discovery using Deep Q-Learning and Variational Autoencoders