229 citations · 260 across the 12 of their papers we have counts for
15 papers
Making Analog Training Scale: Co-Designing Mapping, Optimizer, and Converters
Zhaoxian Wu, Tayfun Gokmen, Omobayode Fagbohungbe +2
Analog in-memory computing (AIMC) offers an alternative for model training by executing matrix operations directly where weights are stored. However, scaling AIMC to train modern d…
Optimization under Persistent State-Dependent Bias: Gradient-based Method and Complexity Analysis
Zhaoxian Wu, Quan Xiao, Tayfun Gokmen +1
This paper studies the convergence of stochastic gradient descent (SGD) when the implemented updates are subject to a persistent and state-dependent bias, in which the desired upda…
Dynamic Symmetric Point Tracking: Tackling Non-ideal Reference in Analog In-memory Training
Quan Xiao, Jindan Li, Zhaoxian Wu +2
Analog in-memory computing (AIMC) performs computation directly within resistive crossbar arrays, offering an energy-efficient platform to scale large vision and language models. H…
In-memory Training on Analog Devices with Limited Conductance States via Multi-tile Residual Learning
Jindan Li, Zhaoxian Wu, Gaowen Liu +2
Analog in-memory computing (AIMC) accelerators enable efficient deep neural network computation directly within memory using resistive crossbar arrays, where model parameters are r…
Analog In-memory Training on General Non-ideal Resistive Elements: The Impact of Response Functions
Zhaoxian Wu, Quan Xiao, Tayfun Gokmen +2
As the economic and environmental costs of training and deploying large vision or language models increase dramatically, analog in-memory computing (AIMC) emerges as a promising en…
Single-Timescale Multi-Sequence Stochastic Approximation Without Fixed Point Smoothness: Theories and Applications
Yue Huang, Zhaoxian Wu, Shiqian Ma +1
Stochastic approximation (SA) that involves multiple coupled sequences, known as multiple-sequence SA (MSSA), finds diverse applications in the fields of signal processing and mach…
On the Convergence Theory of Pipeline Gradient-based Analog In-memory Training
Zhaoxian Wu, Quan Xiao, Tayfun Gokmen +3
Aiming to accelerate the training of large deep neural networks (DNN) in an energy-efficient way, analog in-memory computing (AIMC) emerges as a solution with immense potential. AI…
Towards Exact Gradient-based Training on Analog In-memory Computing
Zhaoxian Wu, Tayfun Gokmen, Malte J. Rasch +1
Given the high economic and environmental costs of using large vision or language models, analog in-memory accelerators present a promising solution for energy-efficient AI. While…
On the Trade-off between Flatness and Optimization in Distributed Learning
Ying Cao, Zhaoxian Wu, Kun Yuan +1
This paper proposes a theoretical framework to evaluate and compare the performance of stochastic gradient algorithms for distributed learning in relation to their behavior around…
Byzantine-Robust Distributed Online Learning: Taming Adversarial Participants in An Adversarial Environment
Xingrong Dong, Zhaoxian Wu, Qing Ling +1
This paper studies distributed online learning under Byzantine attacks. The performance of an online learning algorithm is often characterized by (adversarial) regret, which evalua…
Byzantine-Resilient Decentralized Stochastic Optimization with Robust Aggregation Rules
Zhaoxian Wu, Tianyi Chen, Qing Ling
This paper focuses on decentralized stochastic optimization in the presence of Byzantine attacks. During the optimization process, an unknown number of malfunctioning or malicious…
Byzantine-Resilient Decentralized TD Learning with Linear Function Approximation
Zhaoxian Wu, Han Shen, Tianyi Chen +1
This paper considers the policy evaluation problem in a multi-agent reinforcement learning (MARL) environment over decentralized and directed networks. The focus is on decentralize…
Byzantine-Robust Variance-Reduced Federated Learning over Distributed Non-i.i.d. Data
Jie Peng, Zhaoxian Wu, Qing Ling +1
We consider the federated learning problem where data on workers are not independent and identically distributed (i.i.d.). During the learning process, an unknown number of Byzanti…
Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attacks
Zhaoxian Wu, Qing Ling, Tianyi Chen +1
This paper deals with distributed finite-sum optimization for learning over networks in the presence of malicious Byzantine attacks. To cope with such attacks, most resilient appro…
Communication-Censored Distributed Stochastic Gradient Descent
Weiyu Li, Tianyi Chen, Liping Li +2
This paper develops a communication-efficient algorithm to solve the stochastic optimization problem defined over a distributed network, aiming at reducing the burdensome communica…