MRM3: Machine Readable ML Model Metadata

Abstract

As the complexity and number of machine learning (ML) models grows, well-documented ML models are essential for developers and companies to use or adapt them to their specific use cases. Model metadata, already present in unstructured format as model cards in online repositories such as Hugging Face, could be more structured and machine readable while also incorporating environmental impact metrics such as energy consumption and carbon footprint. Our work extends the existing State of the Art by defining a structured schema for ML model metadata focusing on machine-readable format and support for integration into a knowledge graph (KG) for better organization and querying, enabling a wider set of use cases. Furthermore, we present an example wireless localization model metadata dataset consisting of 22 models trained on 4 datasets, integrated into a Neo4j-based KG with 113 nodes and 199 relations.

References in corpus (9)

• Model Cards for Model Reporting
• Knowledge Graphs
• Compute and Energy Consumption Trends in Deep Learning Inference
• Interactive Model Cards: A Human-Centered Approach to Model Documentation
• Aspirations and Practice of Model Documentation: Moving the Needle with Nudging and Traceability
• A domain-specific language for describing machine learning datasets
• Reusable Templates and Guides For Documenting Datasets and Models for Natural Language Processing and Generation: A Case Study of the HuggingFace and GEM Data and Model Cards
• Croissant: A Metadata Format for ML-Ready Datasets
• An Overview and Solution for Democratizing AI Workflows at the Network Edge