Combining Model-Free Q-Ensembles and Model-Based Approaches for Informed Exploration

Abstract

Q-Ensembles are a model-free approach where input images are fed into different Q-networks and exploration is driven by the assumption that uncertainty is proportional to the variance of the output Q-values obtained. They have been shown to perform relatively well compared to other exploration strategies. Further, model-based approaches, such as encoder-decoder models have been used successfully for next frame prediction given previous frames. This paper proposes to integrate the model-free Q-ensembles and model-based approaches with the hope of compounding the benefits of both and achieving superior exploration as a result. Results show that a model-based trajectory memory approach when combined with Q-ensembles produces superior performance when compared to only using Q-ensembles.

arXiv commentSubmitted to the Thirty-Second Annual Conference on Neural Information Processing Systems (NIPS 2018)

References in corpus (6)

• Efficient Estimation of Word Representations in Vector Space
• Weight Uncertainty in Neural Networks
• Gaussian Processes for Data-Efficient Learning in Robotics and Control
• UCB Exploration via Q-Ensembles
• A Deep Learning Approach for Joint Video Frame and Reward Prediction in Atari Games
• MBMF: Model-Based Priors for Model-Free Reinforcement Learning