9 papers
OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers
Siyuan Li, Jiabao Pan, Yumou Liu +9
Optimizer selection for large-scale model training has become a system-level design decision constrained jointly by compute, memory, tuning budget, and task diversity, yet the land…
TrinityDNA: A Bio-Inspired Foundational Model for Efficient Long-Sequence DNA Modeling
Qirong Yang, Yucheng Guo, Zicheng Liu +7
The modeling of genomic sequences presents unique challenges due to their length and structural complexity. Traditional sequence models struggle to capture long-range dependencies…
MergeMix: A Unified Augmentation Paradigm for Visual and Multi-Modal Understanding
Xin Jin, Siyuan Li, Siyong Jian +2
Vision-language alignment in multi-modal large language models (MLLMs) relies on supervised fine-tuning (SFT) or reinforcement learning (RL). To align multi-modal large language mo…
Life-Code: Central Dogma Modeling with Multi-Omics Sequence Unification
Zicheng Liu, Siyuan Li, Zhiyuan Chen +6
The interactions between DNA, RNA, and proteins are fundamental to biological processes, as illustrated by the central dogma of molecular biology. Although modern biological pre-tr…
MergeDNA: Context-aware Genome Modeling with Dynamic Tokenization through Token Merging
Siyuan Li, Kai Yu, Anna Wang +7
Modeling genomic sequences faces two unsolved challenges: the information density varies widely across different regions, while there is no clearly defined minimum vocabulary unit.…
Rep-MTL: Unleashing the Power of Representation-level Task Saliency for Multi-Task Learning
Zedong Wang, Siyuan Li, Dan Xu
Despite the promise of Multi-Task Learning in leveraging complementary knowledge across tasks, existing multi-task optimization (MTO) techniques remain fixated on resolving conflic…