257 citations · 1k across the 75 of their papers we have counts for
79 papers
Large Stepsizes Federated Learning on Logistic Regression with Linearly Separable Data: The Case of Heterogeneous Devices
Hok Fong Wong, Hoi-To Wai, Chung-Yiu Yau
This paper revisits the distributed learning problem for training a multinomial logistic regression model with the Federated Averaging () algorithm. We concentrate…
From topology learning to graph generation: A unifying perspective
Xiaowen Dong, Hoi-To Wai, Siheng Chen +2
Learning graph structures from data is a fundamental problem that spans a wide range of signal processing and machine learning tasks. While significant effort has been made to tack…
Incremental Aggregation on the Grassmannian for Asynchronous Eigenspace Computation
Xiaolu Wang, Jiang Hu, Hoi-To Wai
We study asynchronous optimization for finite-sum eigenspace computation in heterogeneous distributed systems. The theoretical foundations for asynchronous eigenspace computation r…
Learning Graph Topology with Functional Priors via Bilevel Optimization
Chenyue Zhang, Shangyuan Liu, Hoi-To Wai +1
Learning graph topology of complex networks is challenging due to limited data availability and imprecise data models. Different from prior works that focus on structural priors wi…
EMA-Nesterov: Stabilizing Nesterov's Lookahead for Accelerated Deep Learning Optimization
Chung-Yiu Yau, Dawei Li, Athanasios Glentis +3
Lookahead-based acceleration methods, such as Nesterov's momentum, are widely used in optimization, but they often become unreliable in deep learning training mainly due to stochas…
Revisiting the Constant Stepsize Stochastic Approximation with Decision-Dependent Markovian Noise
Hadi Hadavi, Wenlong Mou, Sergey Samsonov +1
We revisit the convergence analysis of constant stepsize stochastic approximation (SA) with decision-dependent Markovian noise, with a focus on characterizing the stationary bias a…
Decentralized Learning with Dynamically Refined Edge Weights: A Data-Dependent Framework
Rongxing Du, Hoi-To Wai
This paper aims to accelerate decentralized optimization by strategically designing the edge weights used in the agent-to-agent message exchanges. We propose a Dynamic Directed Dec…
Learning Graph from Smooth Signals under Partial Observation: A Robustness Analysis
Hoang-Son Nguyen, Hoi-To Wai
Learning the graph underlying a networked system from nodal signals is crucial to downstream tasks in graph signal processing and machine learning. The presence of hidden nodes who…
Asynchronous and Stochastic Distributed Resource Allocation
Qiang Li, Michal Yemini, Hoi-To Wai
This work proposes and studies the distributed resource allocation problem in asynchronous and stochastic settings. We consider a distributed system with multiple workers and a coo…
Stochastic Gradient Descent with Strategic Querying
Nanfei Jiang, Hoi-To Wai, Mahnoosh Alizadeh
This paper considers a finite-sum optimization problem under first-order queries and investigates the benefits of strategic querying on stochastic gradient-based methods compared t…
Byzantine-Resilient Decentralized Online Resource Allocation
Runhua Wang, Qing Ling, Hoi-To Wai +1
In this paper, we investigate the problem of decentralized online resource allocation in the presence of Byzantine attacks. In this problem setting, some agents may be compromised…
Federated Majorize-Minimization: Beyond Parameter Aggregation
Aymeric Dieuleveut, Gersende Fort, Mahmoud Hegazy +1
This paper proposes a unified approach for designing stochastic optimization algorithms that robustly scale to the federated learning setting. Our work studies a class of Majorize-…
RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models
Quan Wei, Chung-Yiu Yau, Hoi-To Wai +4
Supervised fine-tuning is a standard method for adapting pre-trained large language models (LLMs) to downstream tasks. Quantization has been recently studied as a post-training tec…
Decentralized Stochastic Optimization over Unreliable Networks via Two-timescales Updates
Haoming Liu, Chung-Yiu Yau, Hoi-To Wai
This paper introduces a robust two-timescale compressed primal-dual (TiCoPD) algorithm tailored for decentralized optimization under bandwidth-limited and unreliable channels. By i…
A Two-timescale Primal-dual Algorithm for Decentralized Optimization with Compression
Haoming Liu, Chung-Yiu Yau, Hoi-To Wai
This paper proposes a two-timescale compressed primal-dual (TiCoPD) algorithm for decentralized optimization with improved communication efficiency over prior works on primal-dual…
Multilinear Tensor Low-Rank Approximation for Policy-Gradient Methods in Reinforcement Learning
Sergio Rozada, Hoi-To Wai, Antonio G. Marques
Reinforcement learning (RL) aims to estimate the action to take given a (time-varying) state, with the goal of maximizing a cumulative reward function. Predominantly, there are two…
Network Games Induced Prior for Graph Topology Learning
Chenyue Zhang, Shangyuan Liu, Hoi-To Wai +1
Learning the graph topology of a complex network is challenging due to limited data availability and imprecise data models. A common remedy in existing works is to incorporate prio…
A Stochastic Approximation Approach for Efficient Decentralized Optimization on Random Networks
Chung-Yiu Yau, Haoming Liu, Hoi-To Wai
A challenging problem in decentralized optimization is to develop algorithms with fast convergence on random and time varying topologies under unreliable and bandwidth-constrained…
Tighter Analysis for Decentralized Stochastic Gradient Method: Impact of Data Homogeneity
Qiang Li, Hoi-To Wai
This paper studies the effect of data homogeneity on multi-agent stochastic optimization. We consider the decentralized stochastic gradient (DSGD) algorithm and perform a refined c…
Optimal Pricing for Linear-Quadratic Games with Nonlinear Interaction Between Agents
Jiamin Cai, Chenyue Zhang, Hoi-To Wai
This paper studies a class of network games with linear-quadratic payoffs and externalities exerted through a strictly concave interaction function. This class of game is motivated…
Dual-Delayed Asynchronous SGD for Arbitrarily Heterogeneous Data
Xiaolu Wang, Yuchang Sun, Hoi-To Wai +1
We consider the distributed learning problem with data dispersed across multiple workers under the orchestration of a central server. Asynchronous Stochastic Gradient Descent (SGD)…
On Detecting Low-pass Graph Signals under Partial Observations
Hoang-Son Nguyen, Hoi-To Wai
The application of graph signal processing (GSP) on partially observed graph signals with missing nodes has gained attention recently. This is because processing data from large gr…
Stochastic Optimization Schemes for Performative Prediction with Nonconvex Loss
Qiang Li, Hoi-To Wai
This paper studies a risk minimization problem with decision dependent data distribution. The problem pertains to the performative prediction setting in which a trained model can a…
Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignment
Jiaxiang Li, Siliang Zeng, Hoi-To Wai +3
Aligning human preference and value is an important requirement for contemporary foundation models. State-of-the-art techniques such as Reinforcement Learning from Human Feedback (…