Managing ML Pipelines: Feature Stores and the Coming Wave of Embedding Ecosystems

Abstract

The industrial machine learning pipeline requires iterating on model features, training and deploying models, and monitoring deployed models at scale. Feature stores were developed to manage and standardize the engineer's workflow in this end-to-end pipeline, focusing on traditional tabular feature data. In recent years, however, model development has shifted towards using self-supervised pretrained embeddings as model features. Managing these embeddings and the downstream systems that use them introduces new challenges with respect to managing embedding training data, measuring embedding quality, and monitoring downstream models that use embeddings. These challenges are largely unaddressed in standard feature stores. Our goal in this tutorial is to introduce the feature store system and discuss the challenges and current solutions to managing these new embedding-centric pipelines.

References in corpus (5)

• Deep Learning Recommendation Model for Personalization and Recommendation Systems
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• Model Patching: Closing the Subgroup Performance Gap with Data Augmentation
• Understanding the Downstream Instability of Word Embeddings
• ARDA: Automatic Relational Data Augmentation for Machine Learning