186 citations · 412 across the 34 of their papers we have counts for
18 papers · 1 filter
Fairness Improves Learning from Noisily Labeled Long-Tailed Data
Jiaheng Wei, Zhaowei Zhu, Gang Niu +4
Both long-tailed and noisily labeled data frequently appear in real-world applications and impose significant challenges for learning. Most prior works treat either problem in an i…
Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks
Shuai Zhang, Meng Wang, Pin-Yu Chen +3
Due to the significant computational challenge of training large-scale graph neural networks (GNNs), various sparse learning techniques have been exploited to reduce memory and sto…
Towards Understanding How Self-training Tolerates Data Backdoor Poisoning
Soumyadeep Pal, Ren Wang, Yuguang Yao +1
Recent studies on backdoor attacks in model training have shown that polluting a small portion of training data is sufficient to produce incorrect manipulated predictions on poison…
CLAWSAT: Towards Both Robust and Accurate Code Models
Jinghan Jia, Shashank Srikant, Tamara Mitrovska +4
We integrate contrastive learning (CL) with adversarial learning to co-optimize the robustness and accuracy of code models. Different from existing works, we show that code obfusca…
Fairness Reprogramming
Guanhua Zhang, Yihua Zhang, Yang Zhang +4
Despite a surge of recent advances in promoting machine Learning (ML) fairness, the existing mainstream approaches mostly require retraining or finetuning the entire weights of the…
Queried Unlabeled Data Improves and Robustifies Class-Incremental Learning
Tianlong Chen, Sijia Liu, Shiyu Chang +2
Class-incremental learning (CIL) suffers from the notorious dilemma between learning newly added classes and preserving previously learned class knowledge. That catastrophic forget…
Data-Efficient Double-Win Lottery Tickets from Robust Pre-training
Tianlong Chen, Zhenyu Zhang, Sijia Liu +3
Pre-training serves as a broadly adopted starting point for transfer learning on various downstream tasks. Recent investigations of lottery tickets hypothesis (LTH) demonstrate suc…
Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Free
Tianlong Chen, Zhenyu Zhang, Yihua Zhang +3
Trojan attacks threaten deep neural networks (DNNs) by poisoning them to behave normally on most samples, yet to produce manipulated results for inputs attached with a particular t…
CryoRL: Reinforcement Learning Enables Efficient Cryo-EM Data Collection
Quanfu Fan, Yilai Li, Yuguang Yao +4
Single-particle cryo-electron microscopy (cryo-EM) has become one of the mainstream structural biology techniques because of its ability to determine high-resolution structures of…
Optimizer Amalgamation
Tianshu Huang, Tianlong Chen, Sijia Liu +3
Selecting an appropriate optimizer for a given problem is of major interest for researchers and practitioners. Many analytical optimizers have been proposed using a variety of theo…
RMSMP: A Novel Deep Neural Network Quantization Framework with Row-wise Mixed Schemes and Multiple Precisions
Sung-En Chang, Yanyu Li, Mengshu Sun +4
This work proposes a novel Deep Neural Network (DNN) quantization framework, namely RMSMP, with a Row-wise Mixed-Scheme and Multi-Precision approach. Specifically, this is the firs…
MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge
Geng Yuan, Xiaolong Ma, Wei Niu +13
Recently, a new trend of exploring sparsity for accelerating neural network training has emerged, embracing the paradigm of training on the edge. This paper proposes a novel Memory…
Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks
Shuai Zhang, Meng Wang, Sijia Liu +2
The \textit{lottery ticket hypothesis} (LTH) states that learning on a properly pruned network (the \textit{winning ticket}) improves test accuracy over the original unpruned netwo…
Sign-MAML: Efficient Model-Agnostic Meta-Learning by SignSGD
Chen Fan, Parikshit Ram, Sijia Liu
We propose a new computationally-efficient first-order algorithm for Model-Agnostic Meta-Learning (MAML). The key enabling technique is to interpret MAML as a bilevel optimization…
Lottery Ticket Preserves Weight Correlation: Is It Desirable or Not?
Ning Liu, Geng Yuan, Zhengping Che +7
In deep model compression, the recent finding "Lottery Ticket Hypothesis" (LTH) (Frankle & Carbin, 2018) pointed out that there could exist a winning ticket (i.e., a properly prune…
The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models
Tianlong Chen, Jonathan Frankle, Shiyu Chang +4
The computer vision world has been re-gaining enthusiasm in various pre-trained models, including both classical ImageNet supervised pre-training and recently emerged self-supervis…
Training Stronger Baselines for Learning to Optimize
Tianlong Chen, Weiyi Zhang, Jingyang Zhou +4
Learning to optimize (L2O) has gained increasing attention since classical optimizers require laborious problem-specific design and hyperparameter tuning. However, there is a gap b…
The Lottery Ticket Hypothesis for Pre-trained BERT Networks
Tianlong Chen, Jonathan Frankle, Shiyu Chang +4
In natural language processing (NLP), enormous pre-trained models like BERT have become the standard starting point for training on a range of downstream tasks, and similar trends…