Reinforcement Mechanism Design for e-commerce
arXiv:1708.07607
Abstract
We study the problem of allocating impressions to sellers in e-commerce websites, such as Amazon, eBay or Taobao, aiming to maximize the total revenue generated by the platform. We employ a general framework of reinforcement mechanism design, which uses deep reinforcement learning to design efficient algorithms, taking the strategic behaviour of the sellers into account. Specifically, we model the impression allocation problem as a Markov decision process, where the states encode the history of impressions, prices, transactions and generated revenue and the actions are the possible impression allocations in each round. To tackle the problem of continuity and high-dimensionality of states and actions, we adopt the ideas of the DDPG algorithm to design an actor-critic policy gradient algorithm which takes advantage of the problem domain in order to achieve convergence and stability. We evaluate our proposed algorithm, coined IA(GRU), by comparing it against DDPG, as well as several natural heuristics, under different rationality models for the sellers - we assume that sellers follow well-known no-regret type strategies which may vary in their degree of sophistication. We find that IA(GRU) outperforms all algorithms in terms of the total revenue.
References in corpus (7)
- Adam: A Method for Stochastic Optimization
- Continuous control with deep reinforcement learning
- Playing Atari with Deep Reinforcement Learning
- Thompson Sampling for Contextual Bandits with Linear Payoffs
- Memory-based control with recurrent neural networks
- Reinforcement Mechanism Design, with Applications to Dynamic Pricing in Sponsored Search Auctions
- Econometrics for Learning Agents