1k citations · 1.5k across the 81 of their papers we have counts for
12 papers · 1 filter
When Muon Meets Task Interference: A Spectral Perspective on Continual Learning and Model Merging
Shangge Liu, Yuehan Yin, Yinghuan Shi +2
Continual learning (CL) and model merging (MM) both aim to obtain a single model that performs well across multiple tasks, challenged respectively by catastrophic forgetting and we…
Towards the Connection between Activation Sparsity and Flat Minima
Ze Peng, Jian Zhang, Lei Qi +2
The observation that activation sparsity emerges in MLP blocks of standardly trained Transformers offers an opportunity to drastically reduce computation costs without sacrificing…
Leveraging Flatness to Improve Information-Theoretic Generalization Bounds for SGD
Ze Peng, Jian Zhang, Yisen Wang +3
Information-theoretic (IT) generalization bounds have been used to study the generalization of learning algorithms. These bounds are intrinsically data- and algorithm-dependent so…
LibContinual: A Comprehensive Library towards Realistic Continual Learning
Wenbin Li, Shangge Liu, Borui Kang +7
A fundamental challenge in Continual Learning (CL) is catastrophic forgetting, where adapting to new tasks degrades the performance on previous ones. While the field has evolved wi…
PG-LBO: Enhancing High-Dimensional Bayesian Optimization with Pseudo-Label and Gaussian Process Guidance
Taicai Chen, Yue Duan, Dong Li +3
Variational Autoencoder based Bayesian Optimization (VAE-BO) has demonstrated its excellent performance in addressing high-dimensional structured optimization problems. However, cu…
Roll With the Punches: Expansion and Shrinkage of Soft Label Selection for Semi-supervised Fine-Grained Learning
Yue Duan, Zhen Zhao, Lei Qi +3
While semi-supervised learning (SSL) has yielded promising results, the more realistic SSL scenario remains to be explored, in which the unlabeled data exhibits extremely high reco…
A Theoretical Explanation of Activation Sparsity through Flat Minima and Adversarial Robustness
Ze Peng, Lei Qi, Yinghuan Shi +1
A recent empirical observation (Li et al., 2022b) of activation sparsity in MLP blocks offers an opportunity to drastically reduce computation costs for free. Although having attri…
Towards Semi-supervised Learning with Non-random Missing Labels
Yue Duan, Zhen Zhao, Lei Qi +3
Semi-supervised learning (SSL) tackles the label missing problem by enabling the effective usage of unlabeled data. While existing SSL methods focus on the traditional setting, a p…
RDA: Reciprocal Distribution Alignment for Robust Semi-supervised Learning
Yue Duan, Lei Qi, Lei Wang +2
In this work, we propose Reciprocal Distribution Alignment (RDA) to address semi-supervised learning (SSL), which is a hyperparameter-free framework that is independent of confiden…
PLACE dropout: A Progressive Layer-wise and Channel-wise Dropout for Domain Generalization
Jintao Guo, Lei Qi, Yinghuan Shi +1
Domain generalization (DG) aims to learn a generic model from multiple observed source domains that generalizes well to arbitrary unseen target domains without further training. Th…
Learning-based Computer-aided Prescription Model for Parkinson's Disease: A Data-driven Perspective
Yinghuan Shi, Wanqi Yang, Kim-Han Thung +5
In this paper, we study a novel problem: "automatic prescription recommendation for PD patients." To realize this goal, we first build a dataset by collecting 1) symptoms of PD pat…
OPML: A One-Pass Closed-Form Solution for Online Metric Learning
Wenbin Li, Yang Gao, Lei Wang +3
To achieve a low computational cost when performing online metric learning for large-scale data, we present a one-pass closed-form solution namely OPML in this paper. Typically, th…