Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEs
arXiv:2006.16210
Abstract
We present two elegant solutions for modeling continuous-time dynamics, in a novel model-based reinforcement learning (RL) framework for semi-Markov decision processes (SMDPs), using neural ordinary differential equations (ODEs). Our models accurately characterize continuous-time dynamics and enable us to develop high-performing policies using a small amount of data. We also develop a model-based approach for optimizing time schedules to reduce interaction rates with the environment while maintaining the near-optimal performance, which is not possible for model-free methods. We experimentally demonstrate the efficacy of our methods across various continuous-time domains.
arXiv commentNeurIPS 2020, 20 pages, 7 figures
References in corpus (23)
- Sequence to Sequence Learning with Neural Networks
- Continuous control with deep reinforcement learning
- Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
- Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models
- Addressing Function Approximation Error in Actor-Critic Methods
- Prioritized Experience Replay
- Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
- Model-Ensemble Trust-Region Policy Optimization
- MOPO: Model-based Offline Policy Optimization
- Benchmarking Model-Based Reinforcement Learning
- GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series
- MOReL : Model-Based Offline Reinforcement Learning
- Latent ODEs for Irregularly-Sampled Time Series
- Recurrent Environment Simulators
- How to train your neural ODE: the world of Jacobian and kinetic regularization
- Exploring Model-based Planning with Policy Networks
- Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control
- Learning Differential Equations that are Easy to Solve
- Model-Augmented Actor-Critic: Backpropagating through Paths
- On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models
- Making Deep Q-learning methods robust to time discretization
- Model-based Lookahead Reinforcement Learning
Cited by in corpus (7)
- Adversarial Machine Learning: Bayesian Perspectives
- STEER: Simple Temporal Regularization For Neural ODEs
- Continuous-Time Model-Based Reinforcement Learning
- Continuous-Depth Neural Models for Dynamic Graph Prediction
- Identifying Decision Points for Safe and Interpretable Reinforcement Learning in Hypotension Treatment
- Medical follow-up optimization: A Monte-Carlo planning strategy
- STR-GODEs: Spatial-Temporal-Ridership Graph ODEs for Metro Ridership Prediction