Hyperparameter Search in Machine Learning

Abstract

We introduce the hyperparameter search problem in the field of machine learning and discuss its main challenges from an optimization perspective. Machine learning methods attempt to build models that capture some element of interest based on given data. Most common learning algorithms feature a set of hyperparameters that must be determined before training commences. The choice of hyperparameters can significantly affect the resulting model's performance, but determining good values can be complex; hence a disciplined, theoretically sound search strategy is essential.

5 pages, accepted for MIC 2015: The XI Metaheuristics International Conference in Agadir, Morocco

References in corpus (5)

• Scikit-learn: Machine Learning in Python
• Sequence to Sequence Learning with Neural Networks
• Practical Bayesian Optimization of Machine Learning Algorithms
• EnsembleSVM: A Library for Ensemble Learning Using Support Vector Machines
• Easy Hyperparameter Search Using Optunity

Cited by in corpus (24)

• On Hyperparameter Optimization of Machine Learning Algorithms: Theory and Practice
• Data-driven polynomial chaos expansion for machine learning regression
• An Overview of Machine Learning Techniques for Radiowave Propagation Modeling
• Distributed intelligence on the Edge-to-Cloud Continuum: A systematic literature review
• Supervised Contrastive Learning
• Evolutionary algorithms for hyperparameter optimization in machine learning for application in high energy physics
• Optimal control of families of quantum gates
• Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap
• VisEvol: Visual Analytics to Support Hyperparameter Search through Evolutionary Optimization
• Subspace clustering without knowing the number of clusters: A parameter free approach
• Data-Driven Dynamical Mean-Field Theory: an error-correction approach to solve the quantum many-body problem using machine learning
• Assessment of Neural Network Augmented Reynolds Averaged Navier Stokes Turbulence Model in Extrapolation Modes
• Embedded hyper-parameter tuning by Simulated Annealing
• Quantum Chemistry Calculations using Energy Derivatives on Quantum Computers
• A Novel Genetic Algorithm with Hierarchical Evaluation Strategy for Hyperparameter Optimisation of Graph Neural Networks
• Automatic Exploration of Machine Learning Experiments on OpenML
• Deep Learning Gauss-Manin Connections
• Robust physics discovery via supervised and unsupervised pattern recognition using the Euler characteristic
• The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanations
• What if Process Predictions are not followed by Good Recommendations? (Technical Report)
• Machine Learning Algorithms for Financial Asset Price Forecasting
• Maximum Entropy competes with Maximum Likelihood
• PipeTune: Pipeline Parallelism of Hyper and System Parameters Tuning for Deep Learning Clusters
• REflex: Flexible Framework for Relation Extraction in Multiple Domains