Online Stochastic Optimization with Multiple Objectives

Abstract

In this paper we propose a general framework to characterize and solve the stochastic optimization problems with multiple objectives underlying many real world learning applications. We first propose a projection based algorithm which attains an $O(T^{-1/3})$ convergence rate. Then, by leveraging on the theory of Lagrangian in constrained optimization, we devise a novel primal-dual stochastic approximation algorithm which attains the optimal convergence rate of $O(T^{-1/2})$ for general Lipschitz continuous objectives.

NIPS Workshop on Optimization for Machine Learning

References in corpus (3)

• Making Gradient Descent Optimal for Strongly Convex Stochastic Optimization
• Trading Regret for Efficiency: Online Convex Optimization with Long Term Constraints
• Learning From An Optimization Viewpoint

Cited by in corpus (3)

• Stochastic Successive Convex Approximation for Non-Convex Constrained Stochastic Optimization
• Random constraint sampling and duality for convex optimization
• An Inexact Primal-Dual Algorithm for Semi-Infinite Programming