Online convex optimization and no-regret learning: Algorithms, guarantees and applications

Abstract

Spurred by the enthusiasm surrounding the "Big Data" paradigm, the mathematical and algorithmic tools of online optimization have found widespread use in problems where the trade-off between data exploration and exploitation plays a predominant role. This trade-off is of particular importance to several branches and applications of signal processing, such as data mining, statistical inference, multimedia indexing and wireless communications (to name but a few). With this in mind, the aim of this tutorial paper is to provide a gentle introduction to online optimization and learning algorithms that are asymptotically optimal in hindsight - i.e., they approach the performance of a virtual algorithm with unlimited computational power and full knowledge of the future, a property known as no-regret. Particular attention is devoted to identifying the algorithms' theoretical performance guarantees and to establish links with classic optimization paradigms (both static and stochastic). To allow a better understanding of this toolbox, we provide several examples throughout the tutorial ranging from metric learning to wireless resource allocation problems.

arXiv comment34 pages, 15 figures

References in corpus (3)

• A Survey on Metric Learning for Feature Vectors and Structured Data
• Distributed stochastic optimization via matrix exponential learning
• Parallelized Structures for MIMO FBMC under Strong Channel Frequency Selectivity

Cited by in corpus (5)

• Pricing Mechanism for Resource Sustainability in Competitive Online Learning Multi-Agent Systems
• Learning to Cache With No Regrets
• Optimizing Adaptive Video Streaming in Mobile Networks via Online Learning
• Solving reward-collecting problems with UAVs: a comparison of online optimization and Q-learning
• Bregman algorithms for mixed-strategy generalized Nash equilibrium seeking in a class of mixed-integer games