ParamILS: An Automatic Algorithm Configuration Framework

Abstract

The identification of performance-optimizing parameter settings is an important part of the development and application of algorithms. We describe an automatic framework for this algorithm configuration problem. More formally, we provide methods for optimizing a target algorithm's performance on a given class of problem instances by varying a set of ordinal and/or categorical parameters. We review a family of local-search-based algorithm configuration procedures and present novel techniques for accelerating them by adaptively limiting the time spent for evaluating individual configurations. We describe the results of a comprehensive experimental evaluation of our methods, based on the configuration of prominent complete and incomplete algorithms for SAT. We also present what is, to our knowledge, the first published work on automatically configuring the CPLEX mixed integer programming solver. All the algorithms we considered had default parameter settings that were manually identified with considerable effort. Nevertheless, using our automated algorithm configuration procedures, we achieved substantial and consistent performance improvements.

References in corpus (1)

• A Bayesian Approach to Tackling Hard Computational Problems

Cited by in corpus (62)

• A Tutorial on the Design, Experimentation and Application of Metaheuristic Algorithms to Real-World Optimization Problems
• An Easy-to-use Real-world Multi-objective Optimization Problem Suite
• Best practices for comparing optimization algorithms
• A Prescription of Methodological Guidelines for Comparing Bio-inspired Optimization Algorithms
• Speeding up the Hyperparameter Optimization of Deep Convolutional Neural Networks
• Automated Reinforcement Learning (AutoRL): A Survey and Open Problems
• A meta-learning recommender system for hyperparameter tuning: predicting when tuning improves SVM classifiers
• Theory of Parameter Control for Discrete Black-Box Optimization: Provable Performance Gains Through Dynamic Parameter Choices
• SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization
• Benchmarking in Optimization: Best Practice and Open Issues
• Automated Machine Learning on Graphs: A Survey
• Reinforcement learning based parameters adaption method for particle swarm optimization
• Efficient Multi-Start Strategies for Local Search Algorithms
• Easy Hyperparameter Search Using Optunity
• Towards Green Automated Machine Learning: Status Quo and Future Directions
• Solving Mixed Integer Programs Using Neural Networks
• Multi-Objectivizing Software Configuration Tuning (for a single performance concern)
• Neural network reconstructions for the Hubble parameter, growth rate and distance modulus
• VisEvol: Visual Analytics to Support Hyperparameter Search through Evolutionary Optimization
• SAT-based Encodings for Optimal Decision Trees with Explicit Paths
• Automated Configuration of Genetic Algorithms by Tuning for Anytime Performance
• Neural Networks Optimized by Genetic Algorithms in Cosmology
• Optimal Decision Trees for the Algorithm Selection Problem: Integer Programming Based Approaches
• Tuning metaheuristics by sequential optimization of regression models
• Accuracy Can Lie: On the Impact of Surrogate Model in Configuration Tuning
• Qualities, challenges and future of genetic algorithms: a literature review
• Adapting Multi-objectivized Software Configuration Tuning
• Simulating Non Stationary Operators in Search Algorithms
• Generalization and Completeness of Stochastic Local Search Algorithms
• TunaOil: A Tuning Algorithm Strategy for Reservoir Simulation Workloads
• Using Sequential Runtime Distributions for the Parallel Speedup Prediction of SAT Local Search
• A Comprehensive Survey of Benchmarks for Automated Improvement of Software's Non-Functional Properties
• Global Continuous Optimization with Error Bound and Fast Convergence
• Regularization in Spider-Style Strategy Discovery and Schedule Construction
• Procrastinating with Confidence: Near-Optimal, Anytime, Adaptive Algorithm Configuration
• Amazon SageMaker Autopilot: a white box AutoML solution at scale
• Sample Complexity of Tree Search Configuration: Cutting Planes and Beyond
• From Shallow to Deep Interactions Between Knowledge Representation, Reasoning and Machine Learning (Kay R. Amel group)
• Learning to Configure Mathematical Programming Solvers by Mathematical Programming
• Online Speedup Learning for Optimal Planning
• Few-shots Parallel Algorithm Portfolio Construction via Co-evolution
• Encoding Selection for Solving Hamiltonian Cycle Problems with ASP
• Automatic Construction of Parallel Portfolios via Explicit Instance Grouping
• A learning-based mathematical programming formulation for the automatic configuration of optimization solvers
• Analyzing Adaptive Parameter Landscapes in Parameter Adaptation Methods for Differential Evolution
• Learning Interpretable Error Functions for Combinatorial Optimization Problem Modeling
• Practical Transfer Learning for Bayesian Optimization
• On Performance Estimation in Automatic Algorithm Configuration
• Generalized Early Stopping in Evolutionary Direct Policy Search
• On the Impact of the Cutoff Time on the Performance of Algorithm Configurators
• Practical and sample efficient zero-shot HPO
• Neural Model-based Optimization with Right-Censored Observations
• Multi-Task Multicriteria Hyperparameter Optimization
• Efficient arc-flow formulations for makespan minimisation on parallel machines with a common server
• The Algorithm Configuration Problem
• Learning to Schedule Heuristics in Branch-and-Bound
• Automatically Tailoring Static Analysis to Custom Usage Scenarios
• Refined bounds for algorithm configuration: The knife-edge of dual class approximability
• Automated Machine Learning, Bounded Rationality, and Rational Metareasoning
• D-VAL: An automatic functional equivalence validation tool for planning domain models
• Leveraging Benchmarking Data for Informed One-Shot Dynamic Algorithm Selection
• Markov Chain methods for the bipartite Boolean quadratic programming problem