Systematic Generalisation through Task Temporal Logic and Deep Reinforcement Learning

Abstract

This work introduces a neuro-symbolic agent that combines deep reinforcement learning (DRL) with temporal logic (TL) to achieve systematic zero-shot, i.e., never-seen-before, generalisation of formally specified instructions. In particular, we present a neuro-symbolic framework where a symbolic module transforms TL specifications into a form that helps the training of a DRL agent targeting generalisation, while a neural module learns systematically to solve the given tasks. We study the emergence of systematic learning in different settings and find that the architecture of the convolutional layers is key when generalising to new instructions. We also provide evidence that systematic learning can emerge with abstract operators such as negation when learning from a few training examples, which previous research have struggled with.

References in corpus (10)

• Neural Architecture Search: A Survey
• Solving Rubik's Cube with a Robot Hand
• Agent57: Outperforming the Atari Human Benchmark
• The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision
• Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning
• Dealing with Sparse Rewards in Reinforcement Learning
• Grounded Language Learning Fast and Slow
• Modular Deep Reinforcement Learning with Temporal Logic Specifications
• A Composable Specification Language for Reinforcement Learning Tasks
• Encoding formulas as deep networks: Reinforcement learning for zero-shot execution of LTL formulas

Cited by in corpus (3)

• Reward Machines: Exploiting Reward Function Structure in Reinforcement Learning
• LTL2Action: Generalizing LTL Instructions for Multi-Task RL
• Lifelong Reinforcement Learning with Temporal Logic Formulas and Reward Machines