32 citations · 97 across the 27 of their papers we have counts for
5 papers · 1 filter
MergeDNA: Context-aware Genome Modeling with Dynamic Tokenization through Token Merging
Siyuan Li, Kai Yu, Anna Wang +7
Modeling genomic sequences faces two unsolved challenges: the information density varies widely across different regions, while there is no clearly defined minimum vocabulary unit.…
TrinityDNA: A Bio-Inspired Foundational Model for Efficient Long-Sequence DNA Modeling
Qirong Yang, Yucheng Guo, Zicheng Liu +7
The modeling of genomic sequences presents unique challenges due to their length and structural complexity. Traditional sequence models struggle to capture long-range dependencies…
Taming LLMs by Scaling Learning Rates with Gradient Grouping
Siyuan Li, Juanxi Tian, Zedong Wang +4
Training large language models (LLMs) poses challenges due to their massive scale and heterogeneous architectures. While adaptive optimizers like AdamW help address gradient variat…
MergeVQ: A Unified Framework for Visual Generation and Representation with Disentangled Token Merging and Quantization
Siyuan Li, Luyuan Zhang, Zedong Wang +8
Masked Image Modeling (MIM) with Vector Quantization (VQ) has achieved great success in both self-supervised pre-training and image generation. However, most existing methods strug…
Life-Code: Central Dogma Modeling with Multi-Omics Sequence Unification
Zicheng Liu, Siyuan Li, Zhiyuan Chen +6
The interactions between DNA, RNA, and proteins are fundamental to biological processes, as illustrated by the central dogma of molecular biology. Although modern biological pre-tr…