Probabilistic Matrix Factorization for Automated Machine Learning

Abstract

In order to achieve state-of-the-art performance, modern machine learning techniques require careful data pre-processing and hyperparameter tuning. Moreover, given the ever increasing number of machine learning models being developed, model selection is becoming increasingly important. Automating the selection and tuning of machine learning pipelines consisting of data pre-processing methods and machine learning models, has long been one of the goals of the machine learning community. In this paper, we tackle this meta-learning task by combining ideas from collaborative filtering and Bayesian optimization. Using probabilistic matrix factorization techniques and acquisition functions from Bayesian optimization, we exploit experiments performed in hundreds of different datasets to guide the exploration of the space of possible pipelines. In our experiments, we show that our approach quickly identifies high-performing pipelines across a wide range of datasets, significantly outperforming the current state-of-the-art.

References in corpus (2)

• Scalable Bayesian Optimization Using Deep Neural Networks
• Bayesian Hyperparameter Optimization for Ensemble Learning

Cited by in corpus (25)

• Automated Machine Learning: State-of-The-Art and Open Challenges
• OBOE: Collaborative Filtering for AutoML Model Selection
• D-VAE: A Variational Autoencoder for Directed Acyclic Graphs
• Towards Green Automated Machine Learning: Status Quo and Future Directions
• FLAML: A Fast and Lightweight AutoML Library
• CAAI -- A Cognitive Architecture to Introduce Artificial Intelligence in Cyber-Physical Production Systems
• How Much Automation Does a Data Scientist Want?
• Automatic Machine Learning by Pipeline Synthesis using Model-Based Reinforcement Learning and a Grammar
• OpenML-Python: an extensible Python API for OpenML
• Fast Task-Aware Architecture Inference
• Meta-Surrogate Benchmarking for Hyperparameter Optimization
• RankML: a Meta Learning-Based Approach for Pre-Ranking Machine Learning Pipelines
• Efficient AutoML Pipeline Search with Matrix and Tensor Factorization
• ARDA: Automatic Relational Data Augmentation for Machine Learning
• Practical Transfer Learning for Bayesian Optimization
• Adaptive Bayesian Linear Regression for Automated Machine Learning
• Solving Constrained CASH Problems with ADMM
• How Low Can We Go: Trading Memory for Error in Low-Precision Training
• Hyperparameter optimization with REINFORCE and Transformers
• Requirements for Developing Robust Neural Networks
• Privileged Zero-Shot AutoML
• Naive Automated Machine Learning -- A Late Baseline for AutoML
• Locality Sensitive Hash Aggregated Nonlinear Neighbourhood Matrix Factorization for Online Sparse Big Data Analysis
• An Extensive Experimental Evaluation of Automated Machine Learning Methods for Recommending Classification Algorithms (Extended Version)
• Recommendation on a Budget: Column Space Recovery from Partially Observed Entries with Random or Active Sampling