A domain-specific language for describing machine learning datasets
arXiv:2207.02848 · doi:10.1016/j.cola.2023.101209
Abstract
Datasets play a central role in the training and evaluation of machine learning (ML) models. But they are also the root cause of many undesired model behaviors, such as biased predictions. To overcome this situation, the ML community is proposing a data-centric cultural shift where data issues are given the attention they deserve, and more standard practices around the gathering and processing of datasets start to be discussed and established. So far, these proposals are mostly high-level guidelines described in natural language and, as such, they are difficult to formalize and apply to particular datasets. In this sense, and inspired by these proposals, we define a new domain-specific language (DSL) to precisely describe machine learning datasets in terms of their structure, data provenance, and social concerns. We believe this DSL will facilitate any ML initiative to leverage and benefit from this data-centric shift in ML (e.g., selecting the most appropriate dataset for a new project or better replicating other ML results). The DSL is implemented as a Visual Studio Code plugin, and it has been published under an open source license.
References in corpus (8)
- Explainability Fact Sheets: A Framework for Systematic Assessment of Explainable Approaches
- CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation
- Behavioral Use Licensing for Responsible AI
- A Data Quality-Driven View of MLOps
- Towards Standardization of Data Licenses: The Montreal Data License
- From Things' Modeling Language (ThingML) to Things' Machine Learning (ThingML2)
- MT-Adapted Datasheets for Datasets: Template and Repository
- StreamBrain: An HPC Framework for Brain-like Neural Networks on CPUs, GPUs and FPGAs