OpenML: networked science in machine learning

Abstract

Many sciences have made significant breakthroughs by adopting online tools that help organize, structure and mine information that is too detailed to be printed in journals. In this paper, we introduce OpenML, a place for machine learning researchers to share and organize data in fine detail, so that they can work more effectively, be more visible, and collaborate with others to tackle harder problems. We discuss how OpenML relates to other examples of networked science and what benefits it brings for machine learning research, individual scientists, as well as students and practitioners.

12 pages, 10 figures

References in corpus (10)

• Galaxy Zoo : Morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey
• Classifier Technology and the Illusion of Progress
• Galaxy Zoo Green Peas: Discovery of A Class of Compact Extremely Star-Forming Galaxies
• Galaxy Zoo: Exploring the Motivations of Citizen Science Volunteers
• Galaxy Zoo : 'Hanny's Voorwerp', a quasar light echo?
• Galaxy Zoo: Passive Red Spirals
• A Candidate Sub-Parsec Supermassive Binary Black Hole System
• Planet Hunters: Assessing the Kepler Inventory of Short Period Planets
• The SDSS SkyServer: Public Access to the Sloan Digital Sky Server Data
• The SDSS SkyServer, Public Access to the Sloan Digital Sky Server Data

Cited by in corpus (217)

• Towards A Rigorous Science of Interpretable Machine Learning
• Hyperparameters and Tuning Strategies for Random Forest
• Convolutional Neural Networks for Classification of Alzheimer's Disease: Overview and Reproducible Evaluation
• Data-driven materials science: status, challenges and perspectives
• Performance evaluation and hyperparameter tuning of statistical and machine-learning models using spatial data
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• A Survey of Deep Meta-Learning
• Interpretable Scientific Discovery with Symbolic Regression: A Review
• Towards Automated Machine Learning: Evaluation and Comparison of AutoML Approaches and Tools
• Hyperparameter Importance Across Datasets
• Multi-Objective Counterfactual Explanations
• Regularized target encoding outperforms traditional methods in supervised machine learning with high cardinality features
• Automated Machine Learning: State-of-The-Art and Open Challenges
• Automated data processing and feature engineering for deep learning and big data applications: a survey
• Where are we now? A large benchmark study of recent symbolic regression methods
• Contemporary Symbolic Regression Methods and their Relative Performance
• Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets
• Visualizing the Feature Importance for Black Box Models
• Large-scale benchmark study of survival prediction methods using multi-omics data
• How Data Scientists Work Together With Domain Experts in Scientific Collaborations: To Find The Right Answer Or To Ask The Right Question?
• Revisiting Deep Learning Models for Tabular Data
• Model-agnostic Feature Importance and Effects with Dependent Features -- A Conditional Subgroup Approach
• Review: Deep Learning in Electron Microscopy
• Automated Machine Learning in Practice: State of the Art and Recent Results
• Codabench: Flexible, Easy-to-Use and Reproducible Benchmarking Platform
• OBOE: Collaborative Filtering for AutoML Model Selection
• Adaptation Strategies for Automated Machine Learning on Evolving Data
• Machine Learning for Condensed Matter Physics
• A meta-learning recommender system for hyperparameter tuning: predicting when tuning improves SVM classifiers
• Scikit-dimension: a Python package for intrinsic dimension estimation
• Better Trees: An empirical study on hyperparameter tuning of classification decision tree induction algorithms
• ConfusionFlow: A model-agnostic visualization for temporal analysis of classifier confusion
• On Characterizing the Capacity of Neural Networks using Algebraic Topology
• Towards Green Automated Machine Learning: Status Quo and Future Directions
• An Open Source AutoML Benchmark
• Automated versus do-it-yourself methods for causal inference: Lessons learned from a data analysis competition
• Towards Feature Selection for Ranking and Classification Exploiting Quantum Annealers
• Master your Metrics with Calibration
• Automating Data Science: Prospects and Challenges
• EvalAI: Towards Better Evaluation Systems for AI Agents
• OpenML: An R Package to Connect to the Machine Learning Platform OpenML
• A User-based Visual Analytics Workflow for Exploratory Model Analysis
• Do Not Trust Additive Explanations
• FLAML: A Fast and Lightweight AutoML Library
• Regularized impurity reduction: Accurate decision trees with complexity guarantees
• Autostacker: A Compositional Evolutionary Learning System
• Modeling Generalization in Machine Learning: A Methodological and Computational Study
• Layered TPOT: Speeding up Tree-based Pipeline Optimization
• Symbolic regression outperforms other models for small data sets
• SDCOR: Scalable Density-based Clustering for Local Outlier Detection in Massive-Scale Datasets
• Efficient End-to-End AutoML via Scalable Search Space Decomposition
• Feature Selection for Classification with QAOA
• Multilabel Classification with R Package mlr
• OpenML Benchmarking Suites
• Incorporating Expert Prior in Bayesian Optimisation via Space Warping
• Closing the AI Knowledge Gap
• DiffPrep: Differentiable Data Preprocessing Pipeline Search for Learning over Tabular Data
• SLISEMAP: Supervised dimensionality reduction through local explanations
• AlphaD3M: Machine Learning Pipeline Synthesis
• FairPrep: Promoting Data to a First-Class Citizen in Studies on Fairness-Enhancing Interventions
• Model-based Asynchronous Hyperparameter and Neural Architecture Search
• Coalitional strategies for efficient individual prediction explanation
• VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition
• Randomization as Regularization: A Degrees of Freedom Explanation for Random Forest Success
• Characterizing classification datasets: a study of meta-features for meta-learning
• Gaussian RBF Centered Kernel Alignment (CKA) in the Large Bandwidth Limit
• PRESISTANT: Learning based assistant for data pre-processing
• OpenML-Python: an extensible Python API for OpenML
• Hyperparameter Learning via Distributional Transfer
• Automatic Machine Learning by Pipeline Synthesis using Model-Based Reinforcement Learning and a Grammar
• Reduced Dilation-Erosion Perceptron for Binary Classification
• Towards AutoML in the presence of Drift: first results
• Yggdrasil Decision Forests: A Fast and Extensible Decision Forests Library
• Multiple Adaptive Bayesian Linear Regression for Scalable Bayesian Optimization with Warm Start
• Random Forest Kernel for High-Dimension Low Sample Size Classification
• Decoding machine learning benchmarks
• PILAE: A Non-gradient Descent Learning Scheme for Deep Feedforward Neural Networks
• Using a thousand optimization tasks to learn hyperparameter search strategies
• When are Deep Networks really better than Decision Forests at small sample sizes, and how?
• TSXplain: Demystification of DNN Decisions for Time-Series using Natural Language and Statistical Features
• Extreme Algorithm Selection With Dyadic Feature Representation
• The Missing Indicator Method: From Low to High Dimensions
• TransBO: Hyperparameter Optimization via Two-Phase Transfer Learning
• Adjusted Measures for Feature Selection Stability for Data Sets with Similar Features
• JSDoop and TensorFlow.js: Volunteer Distributed Web Browser-Based Neural Network Training
• Designing Machine Learning Toolboxes: Concepts, Principles and Patterns
• ML-Schema: Exposing the Semantics of Machine Learning with Schemas and Ontologies
• Conditional Feature Importance for Mixed Data
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption
• Learning to Tune XGBoost with XGBoost
• Demystifying a Dark Art: Understanding Real-World Machine Learning Model Development
• Building a Reproducible Machine Learning Pipeline
• Rethinking Default Values: a Low Cost and Efficient Strategy to Define Hyperparameters
• FAIR AI Models in High Energy Physics
• Efficient Explanations for Knowledge Compilation Languages
• Faster DBSCAN via subsampled similarity queries
• Neural Networks with A La Carte Selection of Activation Functions
• Transfer Learning based Search Space Design for Hyperparameter Tuning
• Enabling Collaborative Data Science Development with the Ballet Framework
• Auto-PyTorch Tabular: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL
• Privacy-preserving Continual Federated Clustering via Adaptive Resonance Theory
• HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPO
• Uniqueness of Medical Data Mining: How the new technologies and data they generate are transforming medicine
• Preprocessor Selection for Machine Learning Pipelines
• OneFlow: One-class flow for anomaly detection based on a minimal volume region
• Assessing the Use of AutoML for Data-Driven Software Engineering
• FIRE: An Optimization Approach for Fast Interpretable Rule Extraction
• Efficient First-Order Contextual Bandits: Prediction, Allocation, and Triangular Discrimination
• Democratizing AI: Non-expert design of prediction tasks
• Semantic Labeling Using a Deep Contextualized Language Model
• Information Losses in Neural Classifiers from Sampling
• Amazon SageMaker Autopilot: a white box AutoML solution at scale
• To Split or Not to Split: The Impact of Disparate Treatment in Classification
• Mind the Gap: Measuring Generalization Performance Across Multiple Objectives
• Iterative missing value imputation based on feature importance
• Meta-Surrogate Benchmarking for Hyperparameter Optimization
• A Review of Meta-level Learning in the Context of Multi-component, Multi-level Evolving Prediction Systems
• Foundations of Symbolic Languages for Model Interpretability
• Label noise detection under the Noise at Random model with ensemble filters
• To tune or not to tune? An Approach for Recommending Important Hyperparameters
• Automated Machine Learning with Monte-Carlo Tree Search
• Empirical Likelihood for Contextual Bandits
• Interpretable Regional Descriptors: Hyperbox-Based Local Explanations
• Benchmark of Data Preprocessing Methods for Imbalanced Classification
• HAMLET: A Hierarchical Agent-based Machine Learning Platform
• Transferable Neural Processes for Hyperparameter Optimization
• Simulation study to evaluate when Plasmode simulation is superior to parametric simulation in estimating the mean squared error of the least squares estimator in linear regression
• ASlib: A Benchmark Library for Algorithm Selection
• Exploiting Reuse in Pipeline-Aware Hyperparameter Tuning
• Meta-Learning to Cluster
• Efficient AutoML Pipeline Search with Matrix and Tensor Factorization
• Shift Happens: Adjusting Classifiers
• On Efficiently Explaining Graph-Based Classifiers
• A Repository for Formal Contexts
• Cross-model Fairness: Empirical Study of Fairness and Ethics Under Model Multiplicity
• Using Sequential Statistical Tests for Efficient Hyperparameter Tuning
• One Line To Rule Them All: Generating LO-Shot Soft-Label Prototypes
• EPP: interpretable score of model predictive power
• HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenML
• AutoML: Exploration v.s. Exploitation
• Employing an Adjusted Stability Measure for Multi-Criteria Model Fitting on Data Sets with Similar Features
• Practical Transfer Learning for Bayesian Optimization
• Benchmarking Multimodal AutoML for Tabular Data with Text Fields
• Uncovering Coresets for Classification With Multi-Objective Evolutionary Algorithms
• PAC-Bayes Bounds for Bandit Problems: A Survey and Experimental Comparison
• High Dimensional Restrictive Federated Model Selection with multi-objective Bayesian Optimization over shifted distributions
• A Comprehensive Study of Shapley Value in Data Analytics
• Squirrel: A Switching Hyperparameter Optimizer
• Random Forests for Adaptive Nearest Neighbor Estimation of Information-Theoretic Quantities
• Automated Machine Learning Techniques for Data Streams
• PipelineProfiler: A Visual Analytics Tool for the Exploration of AutoML Pipelines
• Probabilistic Rollouts for Learning Curve Extrapolation Across Hyperparameter Settings
• Experimental Investigation and Evaluation of Model-based Hyperparameter Optimization
• MeanShift++: Extremely Fast Mode-Seeking With Applications to Segmentation and Object Tracking
• Effective Parallelisation for Machine Learning
• Practical and sample efficient zero-shot HPO
• Imbalanced Regression Pipeline Recommendation
• Neural Model-based Optimization with Right-Censored Observations
• MARVIN: An Open Machine Learning Corpus and Environment for Automated Machine Learning Primitive Annotation and Execution
• DarwinML: A Graph-based Evolutionary Algorithm for Automated Machine Learning
• Component-Wise Boosting of Targets for Multi-Output Prediction
• Federated Learning with Discriminative Naive Bayes Classifier
• A Nonmyopic Approach to Cost-Constrained Bayesian Optimization
• Learning from Aggregate Observations
• Interpretable Meta-Measure for Model Performance
• Online Multi-target regression trees with stacked leaf models
• Hyperparameter Transfer Learning with Adaptive Complexity
• Lifting Interpretability-Performance Trade-off via Automated Feature Engineering
• Iterative Hessian Sketch in Input Sparsity Time
• Joints in Random Forests
• Quantifying With Only Positive Training Data
• Understanding Data Science Lifecycle Provenance via Graph Segmentation and Summarization
• Meta-Learning for Symbolic Hyperparameter Defaults
• CAKE: Confidence in Assignments via K-partition Ensembles
• MementoML: Performance of selected machine learning algorithm configurations on OpenML100 datasets
• Data vs classifiers, who wins?
• Optimizing edge AI models on HPC systems with the edge in the loop
• Bayesian Optimization in Materials Science: A Survey
• Interpretable feature subset selection: A Shapley value based approach
• RSN: Randomized Subspace Newton
• Breiman meets Bellman: Non-Greedy Decision Trees with MDPs
• Capturing and Anticipating User Intents in Data Analytics via Knowledge Graphs
• Probabilistic Modeling for Novelty Detection with Applications to Fraud Identification
• Distance Functions and Normalization Under Stream Scenarios
• Exploring the similarity of medical imaging classification problems
• autoBagging: Learning to Rank Bagging Workflows with Metalearning
• Multiclass Graph-Based Large Margin Classifiers: Unified Approach for Support Vectors and Neural Networks
• Revisiting minimum description length complexity in overparameterized models
• Analyzing Data Selection Techniques with Tools from the Theory of Information Losses
• Unsupervised Instance Selection with Low-Label, Supervised Learning for Outlier Detection
• Pairwise Fairness for Ordinal Regression
• Off-Policy Evaluation via Adaptive Weighting with Data from Contextual Bandits
• Developing Open Source Educational Resources for Machine Learning and Data Science
• Learning the Truth From Only One Side of the Story
• MORE-PLR: multi-output regression employed for partial label ranking
• Federated Learning of AnDE Classifiers
• Improving clustering quality evaluation in noisy Gaussian mixtures
• Dynamic Connected Neural Decision Classifier and Regressor with Dynamic Softing Pruning
• Machine Learning in Epidemiology
• MoRE-LLM: Mixture of Rule Experts Guided by a Large Language Model
• Biquality Learning: a Framework to Design Algorithms Dealing with Closed-Set Distribution Shifts
• ChaCha for Online AutoML
• Effect of Incomplete Meta-dataset on Average Ranking Method
• Frugal Machine Learning
• On tuning deep learning models: a data mining perspective
• Learning Abstract Task Representations
• Transformative Machine Learning
• Naive Automated Machine Learning -- A Late Baseline for AutoML
• Games for Fairness and Interpretability
• Towards Robust Classification with Deep Generative Forests
• Interpretable Anomaly Detection with Mondrian P{ó}lya Forests on Data Streams
• Feature Selection Methods for Cost-Constrained Classification in Random Forests
• An Extensive Experimental Evaluation of Automated Machine Learning Methods for Recommending Classification Algorithms (Extended Version)
• The PHOTON Wizard -- Towards Educational Machine Learning Code Generators
• ABC: Efficient Selection of Machine Learning Configuration on Large Dataset
• Towards a Systematic Development Process of Optimization Methods
• Surrogate Model Based Hyperparameter Tuning for Deep Learning with SPOT