19 citations · 83 across the 17 of their papers we have counts for
12 papers · 1 filter
Rethinking Muon Beyond Pretraining: Spectral Failures and High-Pass Remedies for VLA and RLVR
Chongyu Fan, Gaowen Liu, Mingyi Hong +2
Muon is a matrix-aware optimizer that leverages Newton-Schulz (NS) iterations to enforce spectral gradient orthogonalization by driving all singular values of the momentum matrix t…
Subspace Control: Turning Constrained Model Steering into Controllable Spectral Optimization
Yancheng Huang, Changsheng Wang, Chongyu Fan +7
Foundation models, such as large language models (LLMs), are powerful but often require customization before deployment to satisfy practical constraints such as safety, privacy, an…
Tensor-Compressed Back-Propagation-Free Training for (Physics-Informed) Neural Networks
Yequan Zhao, Xinling Yu, Zhixiong Chen +3
Backward propagation (BP) is widely used to compute the gradients in neural network training. However, it is hard to implement BP on edge devices due to the lack of hardware and so…
An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learning
Yihua Zhang, Prashant Khanduri, Ioannis Tsaknakis +3
Recently, bi-level optimization (BLO) has taken center stage in some very exciting developments in the area of signal processing (SP) and machine learning (ML). Roughly speaking, B…
Model Sparsity Can Simplify Machine Unlearning
Jinghan Jia, Jiancheng Liu, Parikshit Ram +5
In response to recent data regulation requirements, machine unlearning (MU) has emerged as a critical process to remove the influence of specific examples from a given model. Altho…
Robustness-preserving Lifelong Learning via Dataset Condensation
Jinghan Jia, Yihua Zhang, Dogyoon Song +2
Lifelong learning (LL) aims to improve a predictive model as the data source evolves continuously. Most work in this learning paradigm has focused on resolving the problem of 'cata…
What Is Missing in IRM Training and Evaluation? Challenges and Solutions
Yihua Zhang, Pranay Sharma, Parikshit Ram +3
Invariant risk minimization (IRM) has received increasing attention as a way to acquire environment-agnostic data representations and predictions, and as a principled solution for…
Advancing Model Pruning via Bi-level Optimization
Yihua Zhang, Yuguang Yao, Parikshit Ram +5
The deployment constraints in practical applications necessitate the pruning of large-scale deep learning models, i.e., promoting their weight sparsity. As illustrated by the Lotte…
Zeroth-Order SciML: Non-intrusive Integration of Scientific Software with Deep Learning
Ioannis Tsaknakis, Bhavya Kailkhura, Sijia Liu +4
Using deep learning (DL) to accelerate and/or improve scientific workflows can yield discoveries that are otherwise impossible. Unfortunately, DL models have yielded limited succes…
Distributed Adversarial Training to Robustify Deep Neural Networks at Scale
Gaoyuan Zhang, Songtao Lu, Yihua Zhang +7
Current deep neural networks (DNNs) are vulnerable to adversarial attacks, where adversarial perturbations to the inputs can change or manipulate classification. To defend against…
How to Robustify Black-Box ML Models? A Zeroth-Order Optimization Perspective
Yimeng Zhang, Yuguang Yao, Jinghan Jia +4
The lack of adversarial robustness has been recognized as an important issue for state-of-the-art machine learning (ML) models, e.g., deep neural networks (DNNs). Thereby, robustif…
Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimization
Yihua Zhang, Guanhua Zhang, Prashant Khanduri +3
Adversarial training (AT) is a widely recognized defense mechanism to gain the robustness of deep neural networks against adversarial attacks. It is built on min-max optimization (…