Using AntiPatterns to avoid MLOps Mistakes
arXiv:2107.00079
Abstract
We describe lessons learned from developing and deploying machine learning models at scale across the enterprise in a range of financial analytics applications. These lessons are presented in the form of antipatterns. Just as design patterns codify best software engineering practices, antipatterns provide a vocabulary to describe defective practices and methodologies. Here we catalog and document numerous antipatterns in financial ML operations (MLOps). Some antipatterns are due to technical errors, while others are due to not having sufficient knowledge of the surrounding context in which ML results are used. By providing a common vocabulary to discuss these situations, our intent is that antipatterns will support better documentation of issues, rapid communication between stakeholders, and faster resolution of problems. In addition to cataloging antipatterns, we describe solutions, best practices, and future directions toward MLOps maturity.
References in corpus (10)
- SMOTE: Synthetic Minority Over-sampling Technique
- Practical Bayesian Optimization of Machine Learning Algorithms
- SQuAD: 100,000+ Questions for Machine Comprehension of Text
- Characterizing Concept Drift
- Deep Reinforcement Learning that Matters
- Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
- On the Importance of Strong Baselines in Bayesian Deep Learning
- HARK Side of Deep Learning -- From Grad Student Descent to Automated Machine Learning
- Bayesian Optimization for Iterative Learning
- Hierarchical Quickest Change Detection via Surrogates