Personalized Optimization with User's Feedback
arXiv:1905.00775 · doi:10.1016/j.automatica.2021.109767
Abstract
This paper develops an online algorithm to solve a time-varying optimization problem with an objective that comprises a known time-varying cost and an unknown function. This problem structure arises in a number of engineering systems and cyber-physical systems where the known function captures time-varying engineering costs, and the unknown function models user's satisfaction; in this context, the objective is to strike a balance between given performance metrics and user's satisfaction. Key challenges related to the problem at hand are related to (1) the time variability of the problem, and (2) the fact that learning of the user's utility function is performed concurrently with the execution of the online algorithm. This paper leverages Gaussian processes (GP) to learn the unknown cost function from noisy functional evaluation and build pertinent upper confidence bounds. Using the GP formalism, the paper then advocates time-varying optimization tools to design an online algorithm that exhibits tracking of the oracle-based optimal trajectory within an error ball, while learning the user's satisfaction function with no-regret. The algorithmic steps are inexact, to account for possible limited computational budgets or real-time implementation considerations. Numerical examples are illustrated based on a problem related to vehicle platooning.
arXiv comment17 pages, 7 figures -- v3 with additional simulation results and comparisons
References in corpus (19)
- Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design
- Gaussian Processes for Data-Efficient Learning in Robotics and Control
- Rates of contraction of posterior distributions based on Gaussian process priors
- Bayesian Reinforcement Learning: A Survey
- Online Distributed Optimization on Dynamic Networks
- Bandit Convex Optimization for Scalable and Dynamic IoT Management
- Safe Learning of Regions of Attraction for Uncertain, Nonlinear Systems with Gaussian Processes
- Online Primal-Dual Methods with Measurement Feedback for Time-Varying Convex Optimization
- Online Learning with Inexact Proximal Online Gradient Descent Algorithms
- Prediction-Correction Algorithms for Time-Varying Constrained Optimization
- An Incentive-Based Online Optimization Framework for Distribution Grids
- Posterior consistency of Gaussian process prior for nonparametric binary regression
- Optimization and Learning with Information Streams: Time-varying Algorithms and Applications
- Decentralized Online Learning with Kernels
- Frequentist coverage and sup-norm convergence rate in Gaussian process regression
- Cognitive Model Priors for Predicting Human Decisions
- Time-Varying Gaussian Process Bandit Optimization
- A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learning
- Recovering Markov Models from Closed-Loop Data
Cited by in corpus (6)
- Probabilistic design of optimal sequential decision-making algorithms in learning and control
- Online Optimization of Dynamical Systems with Deep Learning Perception
- Personalized incentives as feedback design in generalized Nash equilibrium problems
- Feedback-Based Optimization with Sub-Weibull Gradient Errors and Intermittent Updates
- A Stochastic Operator Framework for Optimization and Learning with Sub-Weibull Errors
- Achievement and Fragility of Long-term Equitability