2 citations · 6 across the 16 of their papers we have counts for
4 papers · 1 filter
Investigating Data Pruning for Pretraining Biological Foundation Models at Scale
Yifan Wu, Jiyue Jiang, Xichen Ye +9
Biological foundation models (BioFMs), pretrained on large-scale biological sequences, have recently shown strong potential in providing meaningful representations for diverse down…
TreeSynth: Synthesizing Diverse Data from Scratch via Tree-Guided Subspace Partitioning
Sheng Wang, Pengan Chen, Jingqi Zhou +7
Model customization necessitates high-quality and diverse datasets, but acquiring such data remains time-consuming and labor-intensive. Despite the great potential of large languag…
MoS: Unleashing Parameter Efficiency of Low-Rank Adaptation with Mixture of Shards
Sheng Wang, Liheng Chen, Pengan Chen +5
The rapid scaling of large language models necessitates more lightweight finetuning methods to reduce the explosive GPU memory overhead when numerous customized models are served s…
PRoLoRA: Partial Rotation Empowers More Parameter-Efficient LoRA
Sheng Wang, Boyang Xue, Jiacheng Ye +4
With the rapid scaling of large language models (LLMs), serving numerous low-rank adaptations (LoRAs) concurrently has become increasingly impractical, leading to unaffordable cost…