6 papers
Extending Precipitation Nowcasting Horizons via Spectral Fusion of Radar Observations and Foundation Model Priors
Yuze Qin, Qingyong Li, Zhiqing Guo +3
Precipitation nowcasting is critical for disaster mitigation and aviation safety. However, radar-only models frequently suffer from a lack of large-scale atmospheric context, leadi…
Towards Minimizing Feature Drift in Model Merging: Layer-wise Task Vector Fusion for Adaptive Knowledge Integration
Wenju Sun, Qingyong Li, Wen Wang +3
Multi-task model merging aims to consolidate knowledge from multiple fine-tuned task-specific experts into a unified model while minimizing performance degradation. Existing method…
CAT Merging: A Training-Free Approach for Resolving Conflicts in Model Merging
Wenju Sun, Qingyong Li, Yangli-ao Geng +1
Multi-task model merging offers a promising paradigm for integrating multiple expert models into a unified model without additional training. Existing state-of-the-art techniques,…
PM-MOE: Mixture of Experts on Private Model Parameters for Personalized Federated Learning
Yu Feng, Yangli-ao Geng, Yifan Zhu +7
Federated learning (FL) has gained widespread attention for its privacy-preserving and collaborative learning capabilities. Due to significant statistical heterogeneity, traditiona…
Task Arithmetic in Trust Region: A Training-Free Model Merging Approach to Navigate Knowledge Conflicts
Wenju Sun, Qingyong Li, Wen Wang +2
Multi-task model merging offers an efficient solution for integrating knowledge from multiple fine-tuned models, mitigating the significant computational and storage demands associ…
xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein
Bo Chen, Xingyi Cheng, Pan Li +12
Protein language models have shown remarkable success in learning biological information from protein sequences. However, most existing models are limited by either autoencoding or…