LADDER: A Human-Level Bidding Agent for Large-Scale Real-Time Online Auctions
arXiv:1708.05565
Abstract
We present LADDER, the first deep reinforcement learning agent that can successfully learn control policies for large-scale real-world problems directly from raw inputs composed of high-level semantic information. The agent is based on an asynchronous stochastic variant of DQN (Deep Q Network) named DASQN. The inputs of the agent are plain-text descriptions of states of a game of incomplete information, i.e. real-time large scale online auctions, and the rewards are auction profits of very large scale. We apply the agent to an essential portion of JD's online RTB (real-time bidding) advertising business and find that it easily beats the former state-of-the-art bidding policy that had been carefully engineered and calibrated by human experts: during JD.com's June 18th anniversary sale, the agent increased the company's ads revenue from the portion by more than 50%, while the advertisers' ROI (return on investment) also improved significantly.
arXiv comment8 pages, 12 figures
References in corpus (3)
Cited by in corpus (5)
- Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising
- Deep Reinforcement Learning for Sponsored Search Real-time Bidding
- MoTiAC: Multi-Objective Actor-Critics for Real-Time Bidding
- Learning to Advertise for Organic Traffic Maximization in E-Commerce Product Feeds
- CIA-Towards a Unified Marketing Optimization Framework for e-Commerce Sponsored Search