110 citations · 269 across the 11 of their papers we have counts for
Showing 2021Show all
3 papers · 1 filter
cs.LG2021★ 18 cited
M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining
Junyang Lin, An Yang, Jinze Bai +9
Recent expeditious developments in deep learning algorithms, distributed training, and even hardware design for large models have enabled training extreme-scale models, say GPT-3 a…
q-bio.QM2021
Combating small molecule aggregation with machine learning
Kuan Lee, Ann Yang, Yen-Chu Lin +3
Biological screens are plagued by false positive hits resulting from aggregation. Thus, methods to triage small colloidally aggregating molecules (SCAMs) are in high demand. Herein…
cs.CL2021★ 48 cited
M6: A Chinese Multimodal Pretrainer
Junyang Lin, Rui Men, An Yang +22
In this work, we construct the largest dataset for multimodal pretraining in Chinese, which consists of over 1.9TB images and 292GB texts that cover a wide range of domains. We pro…