1 citations · 1 across the 6 of their papers we have counts for
6 papers
Towards Understanding Feature Learning in Parameter Transfer
Hua Yuan, Xuran Meng, Qiufeng Wang +6
Parameter transfer is a central paradigm in transfer learning, enabling knowledge reuse across tasks and domains by sharing model parameters between upstream and downstream models.…
Extracting Multimodal Learngene in CLIP: Unveiling the Multimodal Generalizable Knowledge
Ruiming Chen, Junming Yang, Shiyu Xia +3
CLIP (Contrastive Language-Image Pre-training) has attracted widespread attention for its multimodal generalizable knowledge, which is significant for downstream tasks. However, th…
STHFL: Spatio-Temporal Heterogeneous Federated Learning
Shunxin Guo, Hongsong Wang, Shuxia Lin +2
Federated learning is a new framework that protects data privacy and allows multiple devices to cooperate in training machine learning models. Previous studies have proposed multip…
Exploring Learngene via Stage-wise Weight Sharing for Initializing Variable-sized Models
Shi-Yu Xia, Wenxuan Zhu, Xu Yang +1
In practice, we usually need to build variable-sized models adapting for diverse resource constraints in different application scenarios, where weight initialization is an importan…
Transformer as Linear Expansion of Learngene
Shiyu Xia, Miaosen Zhang, Xu Yang +3
We propose expanding the shared Transformer module to produce and initialize Transformers of varying depths, enabling adaptation to diverse resource constraints. Drawing an analogy…
Building Variable-sized Models via Learngene Pool
Boyu Shi, Shiyu Xia, Xu Yang +3
Recently, Stitchable Neural Networks (SN-Net) is proposed to stitch some pre-trained networks for quickly building numerous networks with different complexity and performance trade…