5 citations · 8 across the 17 of their papers we have counts for
9 papers · 1 filter
Ada2MS: A Hybrid Optimization Algorithm Based on Exponential Mixing of Elementwise and Global Second-Moment Estimates
Meng Zhu, Quan Xiao, Weidong Min
Optimization algorithms are core methods by which machine learning models iteratively minimize loss functions, update parameters, learn from data, and improve performance. Momentum…
Dynamic Symmetric Point Tracking: Tackling Non-ideal Reference in Analog In-memory Training
Quan Xiao, Jindan Li, Zhaoxian Wu +2
Analog in-memory computing (AIMC) performs computation directly within resistive crossbar arrays, offering an energy-efficient platform to scale large vision and language models. H…
Bilevel Data Curation for LLM Fine-tuning: Offline Selection and Online Self-Refining Generation
Quan Xiao, Yutong Xuan, Gaowen Liu +2
Supervised fine-tuning (SFT) datasets are critical to the downstream performance of large language models, yet they often contain low-quality or harmful question-response pairs. To…
AdamNX: An Adam improvement algorithm based on a novel exponential decay mechanism for the second-order moment estimate
Meng Zhu, Quan Xiao, Weidong Min
This paper studies the exponential decay mechanism of the second-moment estimate in Adam. We propose AdamNX and a time-varying second-moment decay rate that gradually weakens the c…
Balancing Multi-modal Sensor Learning via Multi-objective Optimization
Heshan Fernando, Quan Xiao, Parikshit Ram +4
Learning-enabled control systems increasingly rely on multiple sensing modalities (e.g., vision, audio, language, etc.) for perception and decision support. A key challenge is that…
A First-order Generative Bilevel Optimization Framework for Diffusion Models
Quan Xiao, Hui Yuan, A F M Saif +4
Diffusion models, which iteratively denoise data samples to synthesize high-quality outputs, have achieved empirical success across domains. However, optimizing these models for do…
Analog In-memory Training on General Non-ideal Resistive Elements: The Impact of Response Functions
Zhaoxian Wu, Quan Xiao, Tayfun Gokmen +2
As the economic and environmental costs of training and deploying large vision or language models increase dramatically, analog in-memory computing (AIMC) emerges as a promising en…
On the Convergence Theory of Pipeline Gradient-based Analog In-memory Training
Zhaoxian Wu, Quan Xiao, Tayfun Gokmen +3
Aiming to accelerate the training of large deep neural networks (DNN) in an energy-efficient way, analog in-memory computing (AIMC) emerges as a solution with immense potential. AI…
On Penalty-based Bilevel Gradient Descent Method
Han Shen, Quan Xiao, Tianyi Chen
Bilevel optimization enjoys a wide range of applications in emerging machine learning and signal processing problems such as hyper-parameter optimization, image reconstruction, met…