MLOps Challenges in Multi-Organization Setup: Experiences from Two Real-World Cases

Abstract

The emerging age of connected, digital world means that there are tons of data, distributed to various organizations and their databases. Since this data can be confidential in nature, it cannot always be openly shared in seek of artificial intelligence (AI) and machine learning (ML) solutions. Instead, we need integration mechanisms, analogous to integration patterns in information systems, to create multi-organization AI/ML systems. In this paper, we present two real-world cases. First, we study integration between two organizations in detail. Second, we address scaling of AI/ML to multi-organization context. The setup we assume is that of continuous deployment, often referred to DevOps in software development. When also ML components are deployed in a similar fashion, term MLOps is used. Towards the end of the paper, we list the main observations and draw some final conclusions. Finally, we propose some directions for future work.

2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN) of 43rd International Conference on Software Engineering (ICSE)

Cited by in corpus (6)

• The Pipeline for the Continuous Development of Artificial Intelligence Models -- Current State of Research and Practice
• "We Have No Idea How Models will Behave in Production until Production": How Engineers Operationalize Machine Learning
• Design Patterns for AI-based Systems: A Multivocal Literature Review and Pattern Repository
• Socio-Technical Anti-Patterns in Building ML-Enabled Software: Insights from Leaders on the Forefront
• Maintainability Challenges in ML: A Systematic Literature Review
• Making sense of AI systems development