6 papers
Introduction to Model-Based Derivative-Free Optimization
Lindon Roberts
The field of derivative-free optimization (DFO) studies algorithms for nonlinear optimization that do not rely on the availability of gradient or Hessian information. It is primari…
Polling Set Construction and Worst-Case Complexity for Direct Search under Polyhedral Convex Constraints
Lindon Roberts, Clément W. Royer
Direct search is one of the most popular derivative-free optimization paradigms, that relies on exploring the variable space using polling directions. To analyze and implement dire…
Accuracy and Relationships of Quadratic Models in Derivative-free Optimization
Yiwen Chen, Warren Hare, Lindon Roberts
We study three quadratic models in model-based derivative-free optimization: the minimum norm (MN), minimum Frobenius norm (MFN), and quadratic generalized simplex derivative (QS)…
Bilevel Learning via Inexact Stochastic Gradient Descent
Mohammad Sadegh Salehi, Subhadip Mukherjee, Lindon Roberts +1
Bilevel optimization is a central tool in machine learning for high-dimensional hyperparameter tuning. Its applications are vast; for instance, in imaging it can be used for learni…
Bilevel Learning with Inexact Stochastic Gradients
Mohammad Sadegh Salehi, Subhadip Mukherjee, Lindon Roberts +1
Bilevel learning has gained prominence in machine learning, inverse problems, and imaging applications, including hyperparameter optimization, learning data-adaptive regularizers,…
Projected proximal gradient trust-region algorithm for nonsmooth optimization
Minh N. Dao, Hung M. Phan, Lindon Roberts
We consider trust-region methods for solving optimization problems where the objective is the sum of a smooth, nonconvex function and a nonsmooth, convex regularizer. We extend the…