Model-based Reinforcement Learning for Predictions and Control for Limit Order Books

Abstract

We build a profitable electronic trading agent with Reinforcement Learning that places buy and sell orders in the stock market. An environment model is built only with historical observational data, and the RL agent learns the trading policy by interacting with the environment model instead of with the real-market to minimize the risk and potential monetary loss. Trained in unsupervised and self-supervised fashion, our environment model learned a temporal and causal representation of the market in latent space through deep neural networks. We demonstrate that the trading policy trained entirely within the environment model can be transferred back into the real market and maintain its profitability. We believe that this environment model can serve as a robust simulator that predicts market movement as well as trade impact for further studies.

References in corpus (10)

• Continuous control with deep reinforcement learning
• A Neural Representation of Sketch Drawings
• Action-Conditional Video Prediction using Deep Networks in Atari Games
• Model-Based Reinforcement Learning for Atari
• DeepLOB: Deep Convolutional Neural Networks for Limit Order Books
• Learning Montezuma's Revenge from a Single Demonstration
• Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Traffic
• A Deep Learning Approach for Joint Video Frame and Reward Prediction in Atari Games
• Financial Trading as a Game: A Deep Reinforcement Learning Approach
• Idiosyncrasies and challenges of data driven learning in electronic trading

Cited by in corpus (4)

• The Evolution of Reinforcement Learning in Quantitative Finance: A Survey
• Asynchronous Deep Double Duelling Q-Learning for Trading-Signal Execution in Limit Order Book Markets
• Adversarial recovery of agent rewards from latent spaces of the limit order book
• Multi-Horizon Forecasting for Limit Order Books: Novel Deep Learning Approaches and Hardware Acceleration using Intelligent Processing Units