8 citations · 19 across the 11 of their papers we have counts for
10 papers · 1 filter
When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers
Hongkang Li, Yihua Zhang, Shuai Zhang +3
Task arithmetic refers to editing the pre-trained model by adding a weighted sum of task vectors, each of which is the weight update from the pre-trained model to fine-tuned models…
Learning on Transformers is Provable Low-Rank and Sparse: A One-layer Analysis
Hongkang Li, Meng Wang, Shuai Zhang +2
Efficient training and inference algorithms, such as low-rank adaption and model pruning, have shown impressive performance for learning Transformer-based large foundation models.…
On the Convergence and Sample Complexity Analysis of Deep Q-Networks with -Greedy Exploration
Shuai Zhang, Hongkang Li, Meng Wang +6
This paper provides a theoretical understanding of Deep Q-Network (DQN) with the -greedy exploration in deep reinforcement learning. Despite the tremendous empirical a…
Non-Convex Optimizations for Machine Learning with Theoretical Guarantee: Robust Matrix Completion and Neural Network Learning
Shuai Zhang
Despite the recent development in machine learning, most learning systems are still under the concept of "black box", where the performance cannot be understood and derived. With t…
Patch-level Routing in Mixture-of-Experts is Provably Sample-efficient for Convolutional Neural Networks
Mohammed Nowaz Rabbani Chowdhury, Shuai Zhang, Meng Wang +2
In deep learning, mixture-of-experts (MoE) activates one or few experts (sub-networks) on a per-sample or per-token basis, resulting in significant computation reduction. The recen…
Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks
Shuai Zhang, Meng Wang, Pin-Yu Chen +3
Due to the significant computational challenge of training large-scale graph neural networks (GNNs), various sparse learning techniques have been exploited to reduce memory and sto…
Learning and generalization of one-hidden-layer neural networks, going beyond standard Gaussian data
Hongkang Li, Shuai Zhang, Meng Wang
This paper analyzes the convergence and generalization of training a one-hidden-layer neural network when the input features follow the Gaussian mixture model consisting of a finit…
How does unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis
Shuai Zhang, Meng Wang, Sijia Liu +2
Self-training, a semi-supervised learning algorithm, leverages a large amount of unlabeled data to improve learning when the labeled data are limited. Despite empirical successes,…
Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks
Shuai Zhang, Meng Wang, Sijia Liu +2
The \textit{lottery ticket hypothesis} (LTH) states that learning on a properly pruned network (the \textit{winning ticket}) improves test accuracy over the original unpruned netwo…
Fast Learning of Graph Neural Networks with Guaranteed Generalizability: One-hidden-layer Case
Shuai Zhang, Meng Wang, Sijia Liu +2
Although graph neural networks (GNNs) have made great progress recently on learning from graph-structured data in practice, their theoretical guarantee on generalizability remains…