You Do Not Need a Bigger Boat: Recommendations at Reasonable Scale in a (Mostly) Serverless and Open Stack

Abstract

We argue that immature data pipelines are preventing a large portion of industry practitioners from leveraging the latest research on recommender systems. We propose our template data stack for machine learning at "reasonable scale", and show how many challenges are solved by embracing a serverless paradigm. Leveraging our experience, we detail how modern open source can provide a pipeline processing terabytes of data with limited infrastructure work.

Manuscript version of a work accepted at RecSys 2021 (camera-ready forthcoming)

References in corpus (3)

• A Transformer-based Embedding Model for Personalized Product Search
• Fantastic Embeddings and How to Align Them: Zero-Shot Inference in a Multi-Shop Scenario
• Shopping in the Multiverse: A Counterfactual Approach to In-Session Attribution