Deep Graph Convolutional Reinforcement Learning for Financial Portfolio Management -- DeepPocket

Abstract

Portfolio management aims at maximizing the return on investment while minimizing risk by continuously reallocating the assets forming the portfolio. These assets are not independent but correlated during a short time period. A graph convolutional reinforcement learning framework called DeepPocket is proposed whose objective is to exploit the time-varying interrelations between financial instruments. These interrelations are represented by a graph whose nodes correspond to the financial instruments while the edges correspond to a pair-wise correlation function in between assets. DeepPocket consists of a restricted, stacked autoencoder for feature extraction, a convolutional network to collect underlying local information shared among financial instruments, and an actor-critic reinforcement learning agent. The actor-critic structure contains two convolutional networks in which the actor learns and enforces an investment policy which is, in turn, evaluated by the critic in order to determine the best course of action by constantly reallocating the various portfolio assets to optimize the expected return on investment. The agent is initially trained offline with online stochastic batching on historical data. As new data become available, it is trained online with a passive concept drift approach to handle unexpected changes in their distributions. DeepPocket is evaluated against five real-life datasets over three distinct investment periods, including during the Covid-19 crisis, and clearly outperformed market indexes.

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• Continuous control with deep reinforcement learning
• Semi-Supervised Classification with Graph Convolutional Networks
• The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains
• Neural Message Passing for Quantum Chemistry
• Geometric deep learning: going beyond Euclidean data
• Spectral Networks and Locally Connected Networks on Graphs
• Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering
• Deep Convolutional Networks on Graph-Structured Data
• The leverage effect in financial markets: retarded volatility and market panic
• Chebyshev Polynomial Approximation for Distributed Signal Processing
• A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem
• Impact of Data Normalization on Deep Neural Network for Time Series Forecasting
• Towards identifying the world stock market cross-correlations: DAX versus Dow Jones
• Emerging interdependence between stock values during financial crashes

Cited by in corpus (3)

• The Evolution of Reinforcement Learning in Quantitative Finance: A Survey
• MCI-GRU: Stock Prediction Model Based on Multi-Head Cross-Attention and Improved GRU
• A Survey on Learning from Graphs with Heterophily: Recent Advances and Future Directions