Neural Logic Reinforcement Learning

Abstract

Deep reinforcement learning (DRL) has achieved significant breakthroughs in various tasks. However, most DRL algorithms suffer a problem of generalizing the learned policy which makes the learning performance largely affected even by minor modifications of the training environment. Except that, the use of deep neural networks makes the learned policies hard to be interpretable. To address these two challenges, we propose a novel algorithm named Neural Logic Reinforcement Learning (NLRL) to represent the policies in reinforcement learning by first-order logic. NLRL is based on policy gradient methods and differentiable inductive logic programming that have demonstrated significant advantages in terms of interpretability and generalisability in supervised tasks. Extensive experiments conducted on cliff-walking and blocks manipulation tasks demonstrate that NLRL can induce interpretable policies achieving near-optimal performance while demonstrating good generalisability to environments of different initial states and problem sizes.

Accpeted by ICML2019

References in corpus (6)

• Towards A Rigorous Science of Interpretable Machine Learning
• Trust Region Policy Optimization
• High-Dimensional Continuous Control Using Generalized Advantage Estimation
• A Study on Overfitting in Deep Reinforcement Learning
• Inductive Policy Selection for First-Order MDPs
• TensorLog: Deep Learning Meets Probabilistic DBs

Cited by in corpus (9)

• Neural Collaborative Reasoning
• Human-like generalization in a machine through predicate learning
• A Theory of Relation Learning and Cross-domain Generalization
• Incorporating Relational Background Knowledge into Reinforcement Learning via Differentiable Inductive Logic Programming
• Differentiable Logic Machines
• Neuro-Symbolic Artificial Intelligence: Current Trends
• Neuro-Symbolic Hierarchical Rule Induction
• Deep Inductive Logic Programming meets Reinforcement Learning
• Neural Networks Enhancement with Logical Knowledge