RecSim: A Configurable Simulation Platform for Recommender Systems

Abstract

We propose RecSim, a configurable platform for authoring simulation environments for recommender systems (RSs) that naturally supports sequential interaction with users. RecSim allows the creation of new environments that reflect particular aspects of user behavior and item structure at a level of abstraction well-suited to pushing the limits of current reinforcement learning (RL) and RS techniques in sequential interactive recommendation problems. Environments can be easily configured that vary assumptions about: user preferences and item familiarity; user latent state and its dynamics; and choice models and other user response behavior. We outline how RecSim offers value to RL and RS researchers and practitioners, and how it can serve as a vehicle for academic-industrial collaboration.

References in corpus (19)

• TensorFlow: A system for large-scale machine learning
• Empirical Analysis of Predictive Algorithms for Collaborative Filtering
• A Contextual-Bandit Approach to Personalized News Article Recommendation
• Soft Actor-Critic Algorithms and Applications
• A Neural Conversational Model
• Deep Reinforcement Learning for Page-wise Recommendations
• Further Optimal Regret Bounds for Thompson Sampling
• Challenges of Real-World Reinforcement Learning
• Dopamine: A Research Framework for Deep Reinforcement Learning
• Horizon: Facebook's Open Source Applied Reinforcement Learning Platform
• Discrete Sequential Prediction of Continuous Actions for Deep RL
• RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising
• Seq2Slate: Re-ranking and Slate Optimization with RNNs
• Deep Reinforcement Learning with Attention for Slate Markov Decision Processes with High-Dimensional States and Actions
• Natural Environment Benchmarks for Reinforcement Learning
• Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning
• Toward Simulating Environments in Reinforcement Learning Based Recommendations
• Beyond Greedy Ranking: Slate Optimization via List-CVAE
• Conversational Recommender System

Cited by in corpus (29)

• D4RL: Datasets for Deep Data-Driven Reinforcement Learning
• An empirical investigation of the challenges of real-world reinforcement learning
• Measuring Recommender System Effects with Simulated Users
• A Survey of Deep Reinforcement Learning in Recommender Systems: A Systematic Review and Future Directions
• Do Offline Metrics Predict Online Performance in Recommender Systems?
• Black-box Off-policy Estimation for Infinite-Horizon Reinforcement Learning
• RecSim NG: Toward Principled Uncertainty Modeling for Recommender Ecosystems
• Batch-Constrained Distributional Reinforcement Learning for Session-based Recommendation
• FINN.no Slates Dataset: A new Sequential Dataset Logging Interactions, allViewed Items and Click Responses/No-Click for Recommender Systems Research
• AliExpress Learning-To-Rank: Maximizing Online Model Performance without Going Online
• Offline Meta-level Model-based Reinforcement Learning Approach for Cold-Start Recommendation
• Understanding Longitudinal Dynamics of Recommender Systems with Agent-Based Modeling and Simulation
• WES: Agent-based User Interaction Simulation on Real Infrastructure
• On Covariate Shift of Latent Confounders in Imitation and Reinforcement Learning
• T-RECS: A Simulation Tool to Study the Societal Impact of Recommender Systems
• Accelerating Offline Reinforcement Learning Application in Real-Time Bidding and Recommendation: Potential Use of Simulation
• Towards Content Provider Aware Recommender Systems: A Simulation Study on the Interplay between User and Provider Utilities
• Interactive Search Based on Deep Reinforcement Learning
• Problem Learning: Towards the Free Will of Machines
• A Load Balanced Recommendation Approach
• MBCAL: Sample Efficient and Variance Reduced Reinforcement Learning for Recommender Systems
• A survey of recommender systems for energy efficiency in buildings: Principles, challenges and prospects
• Optimal Mixture Weights for Off-Policy Evaluation with Multiple Behavior Policies
• Locality-Sensitive Experience Replay for Online Recommendation
• Towards Creating a Standardized Collection of Simple and Targeted Experiments to Analyze Core Aspects of the Recommender Systems Problem
• Incentives in Two-sided Matching Markets with Prediction-enhanced Preference-formation
• Simulations in Recommender Systems: An industry perspective
• Position Paper on Simulating Privacy Dynamics in Recommender Systems
• Imitate TheWorld: A Search Engine Simulation Platform