Enabling Collaborative Data Science Development with the Ballet Framework

Abstract

While the open-source software development model has led to successful large-scale collaborations in building software systems, data science projects are frequently developed by individuals or small teams. We describe challenges to scaling data science collaborations and present a conceptual framework and ML programming model to address them. We instantiate these ideas in Ballet, a lightweight framework for collaborative, open-source data science through a focus on feature engineering, and an accompanying cloud-based development environment. Using our framework, collaborators incrementally propose feature definitions to a repository which are each subjected to an ML performance evaluation and can be automatically merged into an executable feature engineering pipeline. We leverage Ballet to conduct a case study analysis of an income prediction problem with 27 collaborators, and discuss implications for future designers of collaborative projects.

References in corpus (21)

• Estimating Mutual Information
• API design for machine learning software: experiences from the scikit-learn project
• OpenML: networked science in machine learning
• Data Programming: Creating Large Training Sets, Quickly
• Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI
• Automated Machine Learning: From Principles to Practices
• AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data
• How Data Scientists Work Together With Domain Experts in Scientific Collaborations: To Find The Right Answer Or To Ask The Right Question?
• Machine Learning that Matters
• ATMSeer: Increasing Transparency and Controllability in Automated Machine Learning
• Continuous integration in a social-coding world: Empirical evidence from GitHub. **Updated version with corrections**
• Machine Teaching: A New Paradigm for Building Machine Learning Systems
• Online Feature Selection with Group Structure Analysis
• Towards Feature Engineering at Scale for Data from Massive Open Online Courses
• The Machine Learning Bazaar: Harnessing the ML Ecosystem for Effective System Development
• Hidden in Plain Sight: Obfuscated Strings Threatening Your Privacy
• How Much Automation Does a Data Scientist Want?
• Slice-based Learning: A Programming Model for Residual Learning in Critical Data Slices
• Model Assertions for Monitoring and Improving ML Models
• Fabrik: An Online Collaborative Neural Network Editor
• Testing MCMC code

Cited by in corpus (2)

• Towards Feature Engineering with Human and AI's Knowledge: Understanding Data Science Practitioners' Perceptions in Human&AI-Assisted Feature Engineering Design
• Meeting in the notebook: a notebook-based environment for micro-submissions in data science collaborations