17 papers
Closed-Form Spectral Regularization for Multi-Task Model Merging
Yongxian Wei, Runxi Cheng, Xingxuan Zhang +4
Model merging combines several independently fine-tuned experts into a single multi-task model without any training data, reducing the storage, serving, and decentralized-developme…
Memory Grafting: Scaling Language Model Pre-training via Offline Conditional Memory
Runxi Cheng, Yuchen Guan, Yongxian Wei +7
Scaling conditional memory offers a promising way to increase language-model capacity, but existing methods such as Engram learn large memory tables from scratch during pre-trainin…
Learning to Pose Problems: Reasoning-Driven and Solver-Adaptive Data Synthesis
Yongxian Wei, Yilin Zhao, Zixuan Hu +7
Data synthesis for training large reasoning models offers a scalable alternative to limited, human-curated datasets, enabling the creation of high-quality data. However, existing a…
OmniAlpha: Aligning Transparency-Aware Generation via Multi-Task Unified Reinforcement Learning
Hao Yu, Jinglin Wang, Jiabo Zhan +7
Transparency-aware generation requires modeling not only RGB appearance but also alpha-based opacity and cross-layer composition, which are essential for tasks such as image mattin…
Task-Distributionally Robust Data-Free Meta-Learning
Zixuan Hu, Yongxian Wei, Li Shen +4
Data-Free Meta-Learning (DFML) aims to enable efficient learning of unseen few-shot tasks, by meta-learning from multiple pre-trained models without accessing their original traini…
OptMerge: Unifying Multimodal LLM Capabilities and Modalities via Model Merging
Yongxian Wei, Runxi Cheng, Weike Jin +7
Foundation models update slowly due to resource-intensive training, whereas domain-specific models evolve rapidly between releases. Model merging seeks to combine multiple expert m…