A Data Quality-Driven View of MLOps
arXiv:2102.07750
Abstract
Developing machine learning models can be seen as a process similar to the one established for traditional software development. A key difference between the two lies in the strong dependency between the quality of a machine learning model and the quality of the data used to train or perform evaluations. In this work, we demonstrate how different aspects of data quality propagate through various stages of machine learning development. By performing a joint analysis of the impact of well-known data quality dimensions and the downstream machine learning process, we show that different components of a typical MLOps pipeline can be efficiently designed, providing both a technical and theoretical perspective.
References in corpus (7)
- Snorkel: Rapid Training Data Creation with Weak Supervision
- Generative Adversarial Networks recover features in astrophysical images of galaxies beyond the deconvolution limit
- Using transfer learning to detect galaxy mergers
- The Ladder: A Reliable Leaderboard for Machine Learning Competitions
- A model for AGN variability on multiple timescales
- PSFGAN: a generative adversarial network system for separating quasar point sources and host galaxy light
- RadioGAN - Translations between different radio surveys with generative adversarial networks
Cited by in corpus (7)
- The Effects of Data Quality on Machine Learning Performance on Tabular Data
- The Pipeline for the Continuous Development of Artificial Intelligence Models -- Current State of Research and Practice
- Diagnosing Ensemble Few-Shot Classifiers
- A domain-specific language for describing machine learning datasets
- DC-Check: A Data-Centric AI checklist to guide the development of reliable machine learning systems
- Enabling SQL-based Training Data Debugging for Federated Learning
- Pastprop-RNN: improved predictions of the future by correcting the past