5 papers
Null-Space Filtering for Data-Free Continual Model Merging: Preserving Stability, Promoting Plasticity
Zihuan Qiu, Lei Wang, Yang Cao +7
Data-free continual model merging (DFCMM) aims to fuse independently fine-tuned models into a single backbone that evolves with incoming tasks without accessing task data. This pap…
Controlled LLM Training on Spectral Sphere
Tian Xie, Haoming Luo, Haoyu Tang +9
Scaling large models requires optimization strategies that ensure rapid convergence grounded in stability. Maximal Update Parametrization (P) provides a theoretical…
D3LM: A Discrete DNA Diffusion Language Model for Bidirectional DNA Understanding and Generation
Zhao Yang, Hengchang Liu, Chuan Cao +1
Early DNA foundation models adopted BERT-style training, achieving good performance on DNA understanding tasks but lacking generative capabilities. Recent autoregressive models ena…
Diffusion LMs Can Approximate Optimal Infilling Lengths Implicitly
Hengchang Liu, Zhao Yang, Bing Su
Diffusion language models (DLMs) provide a bidirectional generation framework naturally suited for infilling, yet their performance is constrained by the pre-specified infilling le…
SPACE: Your Genomic Profile Predictor is a Powerful DNA Foundation Model
Zhao Yang, Jiwei Zhu, Bing Su
Inspired by the success of unsupervised pre-training paradigms, researchers have applied these approaches to DNA pre-training. However, we argue that these approaches alone yield s…