Learning Multi-Agent Intention-Aware Communication for Optimal Multi-Order Execution in Finance

Abstract

Order execution is a fundamental task in quantitative finance, aiming at finishing acquisition or liquidation for a number of trading orders of the specific assets. Recent advance in model-free reinforcement learning (RL) provides a data-driven solution to the order execution problem. However, the existing works always optimize execution for an individual order, overlooking the practice that multiple orders are specified to execute simultaneously, resulting in suboptimality and bias. In this paper, we first present a multi-agent RL (MARL) method for multi-order execution considering practical constraints. Specifically, we treat every agent as an individual operator to trade one specific order, while keeping communicating with each other and collaborating for maximizing the overall profits. Nevertheless, the existing MARL algorithms often incorporate communication among agents by exchanging only the information of their partial observations, which is inefficient in complicated financial market. To improve collaboration, we then propose a learnable multi-round communication protocol, for the agents communicating the intended actions with each other and refining accordingly. It is optimized through a novel action value attribution method which is provably consistent with the original learning objective yet more efficient. The experiments on the data from two real-world markets have illustrated superior performance with significantly better collaboration effectiveness achieved by our method.

arXiv commentAccepted in KDD 2023; The website is at https://seqml.github.io/marl4fin

References in corpus (16)

• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents
• Learning Attentional Communication for Multi-Agent Cooperation
• Learning Multiagent Communication with Backpropagation
• Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Games
• AlphaStock: A Buying-Winners-and-Selling-Losers Investment Strategy using Interpretable Deep Reinforcement Attention Networks
• A reinforcement learning extension to the Almgren-Chriss model for optimal trade execution
• Modeling Others using Oneself in Multi-Agent Reinforcement Learning
• Learning to Schedule Communication in Multi-agent Reinforcement Learning
• A Cooperative Multi-Agent Reinforcement Learning Framework for Resource Balancing in Complex Logistics Network
• Qlib: An AI-oriented Quantitative Investment Platform
• Double Deep Q-Learning for Optimal Execution
• Learning Individually Inferred Communication for Multi-Agent Cooperation
• Multi-Agent Deep Reinforcement Learning for Liquidation Strategy Analysis
• Stock market microstructure inference via multi-agent reinforcement learning