4 papers · 1 filter
L2R: Low-Rank and Lipschitz-Controlled Routing for Mixture-of-Experts
Minghao Yang, Ren Togo, Guang Li +2
Mixture-of-Experts (MoE) models scale neural networks by conditionally activating a small subset of experts, where the router plays a central role in determining expert specializat…
Aligning LLMs with Biomedical Knowledge using Balanced Fine-Tuning
Zhenchao Tang, Fang Wang, Haohuai He +12
Engineering LLMs to accelerate life sciences research requires a robust alignment with biomedical knowledge. We observe that biomedical text exhibits a fundamentally different unce…
Multimodal 3D Genome Pre-training
Minghao Yang, Pengteng Li, Yan Liang +6
Deep learning techniques have driven significant progress in various analytical tasks within 3D genomics in computational biology. However, a holistic understanding of 3D genomics…
Decay Pruning Method: Smooth Pruning With a Self-Rectifying Procedure
Minghao Yang, Linlin Gao, Pengyuan Li +3
Current structured pruning methods often result in considerable accuracy drops due to abrupt network changes and loss of information from pruned structures. To address these issues…