Derivative-free optimization methods
arXiv:1904.11585 · doi:10.1017/S0962492919000060
Abstract
In many optimization problems arising from scientific, engineering and artificial intelligence applications, objective and constraint functions are available only as the output of a black-box or simulation oracle that does not provide derivative information. Such settings necessitate the use of methods for derivative-free, or zeroth-order, optimization. We provide a review and perspectives on developments in these methods, with an emphasis on highlighting recent developments and on unifying treatment of such problems in the non-linear optimization and machine learning literature. We categorize methods based on assumed properties of the black-box functions, as well as features of the methods. We first overview the primary setting of deterministic methods applied to unconstrained, non-convex optimization problems where the objective function is defined by a deterministic black-box oracle. We then discuss developments in randomized methods, methods that assume some additional structure about the objective (including convexity, separability and general non-smooth compositions), methods for problems where the output of the black-box oracle is stochastic, and methods for handling different types of constraints.
References in corpus (2)
Cited by in corpus (46)
- Quantum Solver of Contracted Eigenvalue Equations for Scalable Molecular Simulations on Quantum Computing Devices
- Quantum Machine Learning: from physics to software engineering
- Environmental, Social, Governance scores and the Missing pillar -- Why does missing information matter?
- An Accelerated Directional Derivative Method for Smooth Stochastic Convex Optimization
- Recent theoretical advances in decentralized distributed convex optimization
- Multi-fidelity Bayesian Optimization: A Review
- Adaptive Tikhonov strategies for stochastic ensemble Kalman inversion
- Derivative-Free Method For Composite Optimization With Applications To Decentralized Distributed Optimization
- Randomized gradient-free methods in convex optimization
- CSI-Free Position Optimization for Movable Antenna Communication Systems: A Black-Box Optimization Approach
- Optimization and Supervised Machine Learning Methods for Fitting Numerical Physics Models without Derivatives
- Smoothing Methods for Automatic Differentiation Across Conditional Branches
- Optimizing Fault-Tolerant Quality-Guaranteed Sensor Deployments for UAV Localization in Critical Areas via Computational Geometry
- A One-bit, Comparison-Based Gradient Estimator
- A Hamilton-Jacobi-based Proximal Operator
- Derivative-Free Multiobjective Trust Region Descent Method Using Radial Basis Function Surrogate Models
- Applying Bayesian Optimization with Gaussian Process Regression to Computational Fluid Dynamics Problems
- Quantifying uncertainty with ensembles of surrogates for blackbox optimization
- Stochastic trust-region algorithm in random subspaces with convergence and expected complexity analyses
- Manifold Sampling for Optimizing Nonsmooth Nonconvex Compositions
- Derivative-Free Optimization of a Rapid-Cycling Synchrotron
- An interior point method for nonlinear constrained derivative-free optimization
- Designing a Framework for Solving Multiobjective Simulation Optimization Problems
- A Novel Noise-Aware Classical Optimizer for Variational Quantum Algorithms
- A Method for Convex Black-Box Integer Global Optimization
- End-to-End Protocol for High-Quality QAOA Parameters with Few Shots
- Curvature-Aware Derivative-Free Optimization
- Robust Pareto Transistor Sizing of GaN HEMTs for Millimeter-Wave Applications
- Reconstruction of a single-active-electron potential from electron momentum distribution produced by strong-field ionization using optimization technique
- An Inexact Restoration Direct Multisearch Filter Approach to Multiobjective Constrained Derivative-free Optimization
- -fully Quadratic Modeling and its Application in a Random Subspace Derivative-free Method
- A Noise-Aware Scalable Subspace Classical Optimizer for the Quantum Approximate Optimization Algorithm
- Blockwise Optimization for Projective Variational Quantum Dynamics (BLOP-VQD): Algorithm and Implementation for Lattice Systems
- Quantum Global Minimum Finder based on Variational Quantum Search
- A New Linear Programming Approach and a New Backtracking Strategy for Multiple-Gradient Descent in Multi-Objective Optimization
- Small errors in random zeroth-order optimization are imaginary
- From High-SNR Radar Signal to ECG: A Transfer Learning Model with Cardio-Focusing Algorithm for Scenarios with Limited Data
- EnKSGD: A Class Of Preconditioned Black Box Optimization And Inversion Algorithms
- TRFD: A derivative-free trust-region method based on finite differences for composite nonsmooth optimization
- Projection-based curve pattern search for black-box optimization over smooth convex sets
- Convergence guarantees for forward gradient descent in the linear regression model
- Penalty decomposition derivative free method for the minimization of partially separable functions over a convex feasible set
- Combining Gradient Information and Primitive Directions for High-Performance Mixed-Integer Optimization
- Certified Multi-Fidelity Zeroth-Order Optimization
- Learning Gradient Flow: Using Equation Discovery to Accelerate Engineering Optimization
- Numerical Optimization of Planar Nozzle Shapes for Fused Deposition Modeling