4 papers
Monitoring and Observability of Machine Learning Systems: Current Practices and Gaps
Joran Leest, Ilias Gerostathopoulos, Patricia Lago +1
Production machine learning (ML) systems fail silently -- not with crashes, but through wrong decisions. While observability is recognized as critical for ML operations, there is a…
Tracing Distribution Shifts with Causal System Maps
Joran Leest, Ilias Gerostathopoulos, Patricia Lago +1
Monitoring machine learning (ML) systems is hard, with standard practice focusing on detecting distribution shifts rather than their causes. Root-cause analysis often relies on man…
From Tea Leaves to System Maps: A Survey and Framework on Context-aware Machine Learning Monitoring
Joran Leest, Claudia Raibulet, Patricia Lago +1
Machine learning (ML) models in production fail when their broader systems -- from data pipelines to deployment environments -- deviate from training assumptions, not merely due to…
Expert-Driven Monitoring of Operational ML Models
Joran Leest, Claudia Raibulet, Ilias Gerostathopoulos +1
We propose Expert Monitoring, an approach that leverages domain expertise to enhance the detection and mitigation of concept drift in machine learning (ML) models. Our approach sup…