21 citations · 67 across the 17 of their papers we have counts for
7 papers · 1 filter
Adversarial Robust Memory-Based Continual Learner
Xiaoyue Mi, Fan Tang, Zonghan Yang +4
Despite the remarkable advances that have been made in continual learning, the adversarial vulnerability of such methods has not been fully discussed. We delve into the adversarial…
Improved Algorithms for Online Rent Minimization Problem Under Unit-Size Jobs
Enze Sun, Zonghan Yang, Yuhao Zhang
We consider the Online Rent Minimization problem, where online jobs with release times, deadlines, and processing times must be scheduled on machines that can be rented for a fixed…
Bridging the Gap between Decision and Logits in Decision-based Knowledge Distillation for Pre-trained Language Models
Qinhong Zhou, Zonghan Yang, Peng Li +1
Conventional knowledge distillation (KD) methods require access to the internal information of teachers, e.g., logits. However, such information may not always be accessible for la…
Improving Adversarial Robustness of DEQs with Explicit Regulations Along the Neural Dynamics
Zonghan Yang, Peng Li, Tianyu Pang +1
Deep equilibrium (DEQ) models replace the multiple-layer stacking of conventional deep networks with a fixed-point iteration of a single-layer transformation. Having been demonstra…
A Closer Look at the Adversarial Robustness of Deep Equilibrium Models
Zonghan Yang, Tianyu Pang, Yang Liu
Deep equilibrium models (DEQs) refrain from the traditional layer-stacking paradigm and turn to find the fixed point of a single layer. DEQs have achieved promising performance on…
Exploring the Impact of Model Scaling on Parameter-Efficient Tuning
Yusheng Su, Chi-Min Chan, Jiali Cheng +9
Parameter-efficient tuning (PET) methods can effectively drive extremely large pre-trained language models (PLMs) by training only minimal parameters. Different PET methods utilize…
Restricted Orthogonal Gradient Projection for Continual Learning
Zeyuan Yang, Zonghan Yang, Peng Li +1
Continual learning aims to avoid catastrophic forgetting and effectively leverage learned experiences to master new knowledge. Existing gradient projection approaches impose hard c…