Towards Sharing Task Environments to Support Reproducible Evaluations of Interactive Recommender Systems

Abstract

Beyond sharing datasets or simulations, we believe the Recommender Systems (RS) community should share Task Environments. In this work, we propose a high-level logical architecture that will help to reason about the core components of a RS Task Environment, identify the differences between Environments, datasets and simulations; and most importantly, understand what needs to be shared about Environments to achieve reproducible experiments. The work presents itself as valuable initial groundwork, open to discussion and extensions.

Included in the Offline Evaluation for Recommender Systems Workshop (REVEAL'19), collocated with ACM RecSys 2019. REVEAL'19, September 20th, 2019, Copenhagen, Denmark