Asynchronous Optimization over Weakly Coupled Renewal Systems

Abstract

This paper considers optimization over multiple renewal systems coupled by time average constraints. These systems act asynchronously over variable length frames. For each system, at the beginning of each renewal frame, it chooses an action which affects the duration of its own frame, the penalty, and the resource expenditure throughout the frame. The goal is to minimize the overall time average penalty subject to several overall time average resource constraints which couple these systems. This problem has applications to task processing networks, coupled Markov decision processes(MDPs) and so on. We propose a distributed algorithm so that each system can make its own decision after observing a global multiplier which is updated slot-wise. We show that this algorithm satisfies the desired constraints and achieves $\mathcal{O}(\epsilon )$ near optimality with $\mathcal{O}(1/{\epsilon }^2)$ convergence time.

31 pages

References in corpus (7)

• ARock: an Algorithmic Framework for Asynchronous Parallel Coordinate Updates
• An Online Convex Optimization Approach to Dynamic Network Resource Allocation
• Adaptive Online Prediction by Following the Perturbed Leader
• Online Convex Optimization with Time-Varying Constraints
• Algorithms with Logarithmic or Sublinear Regret for Constrained Contextual Bandits
• Stochastic Primal-Dual Methods and Sample Complexity of Reinforcement Learning
• A Probabilistic Sample Path Convergence Time Analysis of Drift-Plus-Penalty Algorithm for Stochastic Optimization

Cited by in corpus (2)

• Online Learning in Weakly Coupled Markov Decision Processes: A Convergence Time Study
• Data Center Server Provision: Distributed Asynchronous Control for Coupled Renewal Systems