257 citations · 1k across the 75 of their papers we have counts for
29 papers · 1 filter
Incremental Aggregation on the Grassmannian for Asynchronous Eigenspace Computation
Xiaolu Wang, Jiang Hu, Hoi-To Wai
We study asynchronous optimization for finite-sum eigenspace computation in heterogeneous distributed systems. The theoretical foundations for asynchronous eigenspace computation r…
Revisiting the Constant Stepsize Stochastic Approximation with Decision-Dependent Markovian Noise
Hadi Hadavi, Wenlong Mou, Sergey Samsonov +1
We revisit the convergence analysis of constant stepsize stochastic approximation (SA) with decision-dependent Markovian noise, with a focus on characterizing the stationary bias a…
Decentralized Learning with Dynamically Refined Edge Weights: A Data-Dependent Framework
Rongxing Du, Hoi-To Wai
This paper aims to accelerate decentralized optimization by strategically designing the edge weights used in the agent-to-agent message exchanges. We propose a Dynamic Directed Dec…
Asynchronous and Stochastic Distributed Resource Allocation
Qiang Li, Michal Yemini, Hoi-To Wai
This work proposes and studies the distributed resource allocation problem in asynchronous and stochastic settings. We consider a distributed system with multiple workers and a coo…
Byzantine-Resilient Decentralized Online Resource Allocation
Runhua Wang, Qing Ling, Hoi-To Wai +1
In this paper, we investigate the problem of decentralized online resource allocation in the presence of Byzantine attacks. In this problem setting, some agents may be compromised…
Decentralized Stochastic Optimization over Unreliable Networks via Two-timescales Updates
Haoming Liu, Chung-Yiu Yau, Hoi-To Wai
This paper introduces a robust two-timescale compressed primal-dual (TiCoPD) algorithm tailored for decentralized optimization under bandwidth-limited and unreliable channels. By i…
A Two-timescale Primal-dual Algorithm for Decentralized Optimization with Compression
Haoming Liu, Chung-Yiu Yau, Hoi-To Wai
This paper proposes a two-timescale compressed primal-dual (TiCoPD) algorithm for decentralized optimization with improved communication efficiency over prior works on primal-dual…
A Stochastic Approximation Approach for Efficient Decentralized Optimization on Random Networks
Chung-Yiu Yau, Haoming Liu, Hoi-To Wai
A challenging problem in decentralized optimization is to develop algorithms with fast convergence on random and time varying topologies under unreliable and bandwidth-constrained…
Tighter Analysis for Decentralized Stochastic Gradient Method: Impact of Data Homogeneity
Qiang Li, Hoi-To Wai
This paper studies the effect of data homogeneity on multi-agent stochastic optimization. We consider the decentralized stochastic gradient (DSGD) algorithm and perform a refined c…
Optimal Pricing for Linear-Quadratic Games with Nonlinear Interaction Between Agents
Jiamin Cai, Chenyue Zhang, Hoi-To Wai
This paper studies a class of network games with linear-quadratic payoffs and externalities exerted through a strictly concave interaction function. This class of game is motivated…
Stochastic Optimization Schemes for Performative Prediction with Nonconvex Loss
Qiang Li, Hoi-To Wai
This paper studies a risk minimization problem with decision dependent data distribution. The problem pertains to the performative prediction setting in which a trained model can a…
Clipped SGD Algorithms for Performative Prediction: Tight Bounds for Clipping Bias and Remedies
Qiang Li, Michal Yemini, Hoi-To Wai
This paper studies the convergence of clipped stochastic gradient descent (SGD) algorithms with decision-dependent data distribution. Our setting is motivated by privacy preserving…
Two-timescale Derivative Free Optimization for Performative Prediction with Markovian Data
Haitong Liu, Qiang Li, Hoi-To Wai
This paper studies the performative prediction problem where a learner aims to minimize the expected loss with a decision-dependent data distribution. Such setting is motivated whe…
Linear Speedup of Incremental Aggregated Gradient Methods on Streaming Data
Xiaolu Wang, Cheng Jin, Hoi-To Wai +1
This paper considers a type of incremental aggregated gradient (IAG) method for large-scale distributed optimization. The IAG method is well suited for the parameter server archite…
Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learning
Aymeric Dieuleveut, Gersende Fort, Eric Moulines +1
Stochastic Approximation (SA) is a classical algorithm that has had since the early days a huge impact on signal processing, and nowadays on machine learning, due to the necessity…
Multi-agent Performative Prediction with Greedy Deployment and Consensus Seeking Agents
Qiang Li, Chung-Yiu Yau, Hoi-To Wai
We consider a scenario where multiple agents are learning a common decision vector from data which can be influenced by the agents' decisions. This leads to the problem of multi-ag…
On the Role of Data Homogeneity in Multi-Agent Non-convex Stochastic Optimization
Qiang Li, Hoi-To Wai
This paper studies the role of data homogeneity on multi-agent optimization. Concentrating on the decentralized stochastic gradient (DSGD) algorithm, we characterize the transient…
An Empirical Study on Compressed Decentralized Stochastic Gradient Algorithms with Overparameterized Models
Arjun Ashok Rao, Hoi-To Wai
This paper considers decentralized optimization with application to machine learning on graphs. The growing size of neural network (NN) models has motivated prior works on decentra…
State Dependent Performative Prediction with Stochastic Approximation
Qiang Li, Hoi-To Wai
This paper studies the performative prediction problem which optimizes a stochastic loss function with data distribution that depends on the decision variable. We consider a settin…
Robust Distributed Optimization With Randomly Corrupted Gradients
Berkay Turan, Cesar A. Uribe, Hoi-To Wai +1
In this paper, we propose a first-order distributed optimization algorithm that is provably robust to Byzantine failures-arbitrary and potentially adversarial behavior, where all t…
A Near-Optimal Algorithm for Stochastic Bilevel Optimization via Double-Momentum
Prashant Khanduri, Siliang Zeng, Mingyi Hong +3
This paper proposes a new algorithm -- the \underline{S}ingle-timescale Do\underline{u}ble-momentum \underline{St}ochastic \underline{A}pprox\underline{i}matio\underline{n} (SUSTAI…
On Robustness of the Normalized Subgradient Method with Randomly Corrupted Subgradients
Berkay Turan, Cesar A. Uribe, Hoi-To Wai +1
Numerous modern optimization and machine learning algorithms rely on subgradient information being trustworthy and hence, they may fail to converge when such information is corrupt…
On the Convergence of Consensus Algorithms with Markovian Noise and Gradient Bias
Hoi-To Wai
This paper presents a finite time convergence analysis for a decentralized stochastic approximation (SA) scheme. The scheme generalizes several algorithms for decentralized machine…
A Two-Timescale Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Critic
Mingyi Hong, Hoi-To Wai, Zhaoran Wang +1
This paper analyzes a two-timescale stochastic algorithm framework for bilevel optimization. Bilevel optimization is a class of problems which exhibit a two-level structure, and it…