Budget Optimization for Sponsored Search: Censored Learning in MDPs

Abstract

We consider the budget optimization problem faced by an advertiser participating in repeated sponsored search auctions, seeking to maximize the number of clicks attained under that budget. We cast the budget optimization problem as a Markov Decision Process (MDP) with censored observations, and propose a learning algorithm based on the wellknown Kaplan-Meier or product-limit estimator. We validate the performance of this algorithm by comparing it to several others on a large set of search auction data from Microsoft adCenter, demonstrating fast convergence to optimal performance.

Appears in Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence (UAI2012)

Cited by in corpus (11)

• Bandits with Knapsacks
• Real-Time Bidding by Reinforcement Learning in Display Advertising
• Bidding Machine: Learning to Bid for Directly Optimizing Profits in Display Advertising
• Learning Theory and Algorithms for Revenue Optimization in Second-Price Auctions with Reserve
• Statistical Arbitrage Mining for Display Advertising
• Thompson Sampling for Budgeted Multi-armed Bandits
• Managing Risk of Bidding in Display Advertising
• Thresholding at the monopoly price: an agnostic way to improve bidding strategies in revenue-maximizing auctions
• Optimizing Sponsored Search Ranking Strategy by Deep Reinforcement Learning
• Learning Best Response Strategies for Agents in Ad Exchanges
• Censored Semi-Bandits for Resource Allocation