MoTiAC: Multi-Objective Actor-Critics for Real-Time Bidding
arXiv:2002.07408
Abstract
Online Real-Time Bidding (RTB) is a complex auction game among which advertisers struggle to bid for ad impressions when a user request occurs. Considering display cost, Return on Investment (ROI), and other influential Key Performance Indicators (KPIs), large ad platforms try to balance the trade-off among various goals in dynamics. To address the challenge, we propose a Multi-ObjecTive Actor-Critics algorithm based on reinforcement learning (RL), named MoTiAC, for the problem of bidding optimization with various goals. In MoTiAC, objective-specific agents update the global network asynchronously with different goals and perspectives, leading to a robust bidding policy. Unlike previous RL models, the proposed MoTiAC can simultaneously fulfill multi-objective tasks in complicated bidding environments. In addition, we mathematically prove that our model will converge to Pareto optimality. Finally, experiments on a large-scale real-world commercial dataset from Tencent verify the effectiveness of MoTiAC versus a set of recent approaches
arXiv commentAccepted in ECML-PKDD 2022. Zhou and Yang made equal contributions
References in corpus (8)
- Continuous control with deep reinforcement learning
- Playing Atari with Deep Reinforcement Learning
- Hybrid Reward Architecture for Reinforcement Learning
- Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising
- LADDER: A Human-Level Bidding Agent for Large-Scale Real-Time Online Auctions
- A Nonparametric Delayed Feedback Model for Conversion Rate Prediction
- Deep Reinforcement Learning for Sponsored Search Real-time Bidding
- Toward negotiable reinforcement learning: shifting priorities in Pareto optimal sequential decision-making