10 papers
Optimization under Persistent State-Dependent Bias: Gradient-based Method and Complexity Analysis
Zhaoxian Wu, Quan Xiao, Tayfun Gokmen +1
This paper studies the convergence of stochastic gradient descent (SGD) when the implemented updates are subject to a persistent and state-dependent bias, in which the desired upda…
Dynamic Symmetric Point Tracking: Tackling Non-ideal Reference in Analog In-memory Training
Quan Xiao, Jindan Li, Zhaoxian Wu +2
Analog in-memory computing (AIMC) performs computation directly within resistive crossbar arrays, offering an energy-efficient platform to scale large vision and language models. H…
In-memory Training on Analog Devices with Limited Conductance States via Multi-tile Residual Learning
Jindan Li, Zhaoxian Wu, Gaowen Liu +2
Analog in-memory computing (AIMC) accelerators enable efficient deep neural network computation directly within memory using resistive crossbar arrays, where model parameters are r…
Analog In-memory Training on General Non-ideal Resistive Elements: The Impact of Response Functions
Zhaoxian Wu, Quan Xiao, Tayfun Gokmen +2
As the economic and environmental costs of training and deploying large vision or language models increase dramatically, analog in-memory computing (AIMC) emerges as a promising en…
Single-Timescale Multi-Sequence Stochastic Approximation Without Fixed Point Smoothness: Theories and Applications
Yue Huang, Zhaoxian Wu, Shiqian Ma +1
Stochastic approximation (SA) that involves multiple coupled sequences, known as multiple-sequence SA (MSSA), finds diverse applications in the fields of signal processing and mach…
On the Convergence Theory of Pipeline Gradient-based Analog In-memory Training
Zhaoxian Wu, Quan Xiao, Tayfun Gokmen +3
Aiming to accelerate the training of large deep neural networks (DNN) in an energy-efficient way, analog in-memory computing (AIMC) emerges as a solution with immense potential. AI…