Monitoring and explainability of models in production

Abstract

The machine learning lifecycle extends beyond the deployment stage. Monitoring deployed models is crucial for continued provision of high quality machine learning enabled services. Key areas include model performance and data monitoring, detecting outliers and data drift using statistical techniques, and providing explanations of historic predictions. We discuss the challenges to successful implementation of solutions in each of these areas with some recent examples of production ready solutions using open source tools.

Workshop on Challenges in Deploying and Monitoring Machine Learning Systems (ICML 2020)

References in corpus (3)

• On Calibration of Modern Neural Networks
• Continual Learning in Practice
• Adversarial Detection and Correction by Matching Prediction Distributions

Cited by in corpus (2)

• Pitfalls of Explainable ML: An Industry Perspective
• ModelCI-e: Enabling Continual Learning in Deep Learning Serving Systems