Online Linear Programming: Dual Convergence, New Algorithms, and Regret Bounds

Abstract

We study an online linear programming (OLP) problem under a random input model in which the columns of the constraint matrix along with the corresponding coefficients in the objective function are generated i.i.d. from an unknown distribution and revealed sequentially over time. Virtually all pre-existing online algorithms were based on learning the dual optimal solutions/prices of the linear programs (LP), and their analyses were focused on the aggregate objective value and solving the packing LP where all coefficients in the constraint matrix and objective are nonnegative. However, two major open questions were: (i) Does the set of LP optimal dual prices learned in the pre-existing algorithms converge to those of the "offline" LP, and (ii) Could the results be extended to general LP problems where the coefficients can be either positive or negative. We resolve these two questions by establishing convergence results for the dual prices under moderate regularity conditions for general LP problems. Specifically, we identify an equivalent form of the dual problem which relates the dual LP with a sample average approximation to a stochastic program. Furthermore, we propose a new type of OLP algorithm, Action-History-Dependent Learning Algorithm, which improves the previous algorithm performances by taking into account the past input data as well as decisions/actions already made. We derive an $O(\log n\log \log n)$ regret bound (under a locally strong convexity and smoothness condition) for the proposed algorithm, against the $O(\sqrt{n})$ bound for typical dual-price learning algorithms, where $n$ is the number of decision variables. Numerical experiments demonstrate the effectiveness of the proposed algorithm and the action-history-dependent design.

References in corpus (5)

• Trading Regret for Efficiency: Online Convex Optimization with Long Term Constraints
• Uniformly bounded regret in the multi-secretary problem
• Online convex optimization for cumulative constraints
• Algorithms with Logarithmic or Sublinear Regret for Constrained Contextual Bandits
• A Primal-dual Learning Algorithm for Personalized Dynamic Pricing with an Inventory Constraint

Cited by in corpus (6)

• Online Stochastic Optimization with Wasserstein Based Non-stationarity
• Near-Optimal Primal-Dual Algorithms for Quantity-Based Network Revenue Management
• Online Resource Allocation with Stochastic Resource Consumption
• The Symmetry between Arms and Knapsacks: A Primal-Dual Approach for Bandits with Knapsacks
• Provably Training Overparameterized Neural Network Classifiers with Non-convex Constraints
• Column-Randomized Linear Programs: Performance Guarantees and Applications