184 citations · 354 across the 43 of their papers we have counts for
47 papers
Strategic Arbitrage in Electricity Markets: A Multi-Agent Policy Optimization Framework
Eli Brock, Srikar Babu Gadipudi, Somayeh Sojoudi +1
Grid-scale storage and load aggregators are growing quickly in wholesale electricity markets. If highly consolidated, these new participants may distort the competitive market equi…
High Probability Complexity Bounds of Trust-Region Stochastic Sequential Quadratic Programming with Heavy-Tailed Noise
Yuchen Fang, Javad Lavaei, Sen Na
In this paper, we consider nonlinear optimization problems with a stochastic objective and deterministic equality constraints. We propose a Trust-Region Stochastic Sequential Quadr…
The landscape of deterministic and stochastic optimal control problems: One-shot Optimization versus Dynamic Programming
Jihun Kim, Yuhao Ding, Yingjie Bi +1
Optimal control problems can be solved via a one-shot (single) optimization or a sequence of optimization using dynamic programming (DP). However, the computation of their global o…
A CMDP-within-online framework for Meta-Safe Reinforcement Learning
Vanshaj Khattar, Yuhao Ding, Bilgehan Sel +2
Meta-reinforcement learning has widely been used as a learning-to-learn framework to solve unseen tasks with limited experience. However, the aspect of constraint violations has no…
Absence of spurious solutions far from ground truth: A low-rank analysis with high-order losses
Ziye Ma, Ying Chen, Javad Lavaei +1
Matrix sensing problems exhibit pervasive non-convexity, plaguing optimization with a proliferation of suboptimal spurious solutions. Avoiding convergence to these critical points…
Tempo Adaptation in Non-stationary Reinforcement Learning
Hyunin Lee, Yuhao Ding, Jongmin Lee +3
We first raise and tackle a ``time synchronization'' issue between the agent and the environment in non-stationary reinforcement learning (RL), a crucial factor hindering its real-…
Algorithmic Regularization in Tensor Optimization: Towards a Lifted Approach in Matrix Sensing
Ziye Ma, Javad Lavaei, Somayeh Sojoudi
Gradient descent (GD) is crucial for generalization in machine learning models, as it induces implicit regularization, promoting compact representations. In this work, we examine t…
Scalable Primal-Dual Actor-Critic Method for Safe Multi-Agent RL with General Utilities
Donghao Ying, Yunkai Zhang, Yuhao Ding +2
We investigate safe multi-agent reinforcement learning, where agents seek to collectively maximize an aggregate sum of local objectives while satisfying their own safety constraint…
No-Regret Learning in Dynamic Competition with Reference Effects Under Logit Demand
Mengzi Amy Guo, Donghao Ying, Javad Lavaei +1
This work is dedicated to the algorithm design in a competitive framework, with the primary goal of learning a stable equilibrium. We consider the dynamic price competition between…
Exact Recovery for System Identification with More Corrupt Data than Clean Data
Baturalp Yalcin, Haixiang Zhang, Javad Lavaei +1
This paper investigates the system identification problem for linear discrete-time systems under adversaries and analyzes two lasso-type estimators. We examine both asymptotic and…
A Hitting Time Analysis for Stochastic Time-Varying Functions with Applications to Adversarial Attacks on Computation of Markov Decision Processes
Ali Yekkehkhany, Han Feng, Donghao Ying +1
Stochastic time-varying optimization is an integral part of learning in which the shape of the function changes over time in a non-deterministic manner. This paper considers multip…
Over-parametrization via Lifting for Low-rank Matrix Sensing: Conversion of Spurious Solutions to Strict Saddle Points
Ziye Ma, Igor Molybog, Javad Lavaei +1
This paper studies the role of over-parametrization in solving non-convex optimization problems. The focus is on the important class of low-rank matrix sensing, where we propose an…
Scalable Multi-Agent Reinforcement Learning with General Utilities
Donghao Ying, Yuhao Ding, Alec Koppel +1
We study the scalable multi-agent reinforcement learning (MARL) with general utilities, defined as nonlinear functions of the team's long-term state-action occupancy measure. The o…
Non-stationary Risk-sensitive Reinforcement Learning: Near-optimal Dynamic Regret, Adaptive Detection, and Separation Design
Yuhao Ding, Ming Jin, Javad Lavaei
We study risk-sensitive reinforcement learning (RL) based on an entropic risk measure in episodic non-stationary Markov decision processes (MDPs). Both the reward functions and the…
Learning of Dynamical Systems under Adversarial Attacks -- Null Space Property Perspective
Han Feng, Baturalp Yalcin, Javad Lavaei
We study the identification of a linear time-invariant dynamical system affected by large-and-sparse disturbances modeling adversarial attacks or faults. Under the assumption that…
Semidefinite Programming versus Burer-Monteiro Factorization for Matrix Sensing
Baturalp Yalcin, Ziye Ma, Javad Lavaei +1
Many fundamental low-rank optimization problems, such as matrix completion, phase synchronization/retrieval, power system state estimation, and robust PCA, can be formulated as the…
Policy-based Primal-Dual Methods for Concave CMDP with Variance Reduction
Donghao Ying, Mengzi Amy Guo, Hyunin Lee +3
We study Concave Constrained Markov Decision Processes (Concave CMDPs) where both the objective and constraints are defined as concave functions of the state-action occupancy measu…
A New Complexity Metric for Nonconvex Rank-one Generalized Matrix Completion
Haixiang Zhang, Baturalp Yalcin, Javad Lavaei +1
In this work, we develop a new complexity metric for an important class of low-rank matrix optimization problems in both symmetric and asymmetric cases, where the metric aims to qu…
Provably Efficient Primal-Dual Reinforcement Learning for CMDPs with Non-stationary Objectives and Constraints
Yuhao Ding, Javad Lavaei
We consider primal-dual-based reinforcement learning (RL) in episodic constrained Markov decision processes (CMDPs) with non-stationary objectives and constraints, which plays a ce…
Factorization Approach for Low-complexity Matrix Completion Problems: Exponential Number of Spurious Solutions and Failure of Gradient Methods
Baturalp Yalcin, Haixiang Zhang, Javad Lavaei +1
It is well-known that the Burer-Monteiro (B-M) factorization approach can efficiently solve low-rank matrix optimization problems under the RIP condition. It is natural to ask whet…
Beyond Exact Gradients: Convergence of Stochastic Soft-Max Policy Gradient Methods with Entropy Regularization
Yuhao Ding, Junzi Zhang, Hyunin Lee +1
Entropy regularization is an efficient technique for encouraging exploration and preventing a premature convergence of (vanilla) policy gradient methods in reinforcement learning (…
On the Global Optimum Convergence of Momentum-based Policy Gradient
Yuhao Ding, Junzi Zhang, Javad Lavaei
Policy gradient (PG) methods are popular and efficient for large-scale reinforcement learning due to their relative stability and incremental nature. In recent years, the empirical…
A Dual Approach to Constrained Markov Decision Processes with Entropy Regularization
Donghao Ying, Yuhao Ding, Javad Lavaei
We study entropy-regularized constrained Markov decision processes (CMDPs) under the soft-max parameterization, in which an agent aims to maximize the entropy-regularized value fun…
Sharp Restricted Isometry Property Bounds for Low-rank Matrix Recovery Problems with Corrupted Measurements
Ziye Ma, Yingjie Bi, Javad Lavaei +1
In this paper, we study a general low-rank matrix recovery problem with linear measurements corrupted by some noise. The objective is to understand under what conditions on the res…