A reinforcement learning extension to the Almgren-Chriss model for optimal trade execution

Abstract

Reinforcement learning is explored as a candidate machine learning technique to enhance existing analytical solutions for optimal trade execution with elements from the market microstructure. Given a volume-to-trade, fixed time horizon and discrete trading periods, the aim is to adapt a given volume trajectory such that it is dynamic with respect to favourable/unfavourable conditions during realtime execution, thereby improving overall cost of trading. We consider the standard Almgren-Chriss model with linear price impact as a candidate base model. This model is popular amongst sell-side institutions as a basis for arrival price benchmark execution algorithms. By training a learning agent to modify a volume trajectory based on the market's prevailing spread and volume dynamics, we are able to improve post-trade implementation shortfall by up to 10.3% on average compared to the base model, based on a sample of stocks and trade sizes in the South African equity market.

arXiv comment6 pages, 2014 IEEE Computational Intelligence for Financial Engineering and Economics conference (accepted)

Cited by in corpus (13)

• Continuous-Time Mean-Variance Portfolio Selection: A Reinforcement Learning Framework
• Double Deep Q-Learning for Optimal Execution
• A Framework for Online Investment Algorithms
• Multi-Agent Deep Reinforcement Learning for Liquidation Strategy Analysis
• Policy Gradient Methods for the Noisy Linear Quadratic Regulator over a Finite Horizon
• Deviations in expected price impact for small transaction volumes under fee restructuring
• Malliavin-Mancino estimators implemented with non-uniform fast Fourier transforms
• Learning Multi-Agent Intention-Aware Communication for Optimal Multi-Order Execution in Finance
• Learning the dynamics of technical trading strategies
• Reinforcement Learning for Quantitative Trading
• The Problem of Calibrating an Agent-Based Model of High-Frequency Trading
• Comparing the market microstructure between two South African exchanges
• Deep equal risk pricing of financial derivatives with non-translation invariant risk measures