306 citations · 700 across the 24 of their papers we have counts for
12 papers · 1 filter
BioMatrix: Towards a Comprehensive Biological Foundation Model Spanning the Modality Matrix of Sequences, Structures, and Language
Qizhi Pei, Zhimeng Zhou, Yi Duan +9
We present BioMatrix, the first multimodal foundation model that natively integrates sequences, structures, and natural language for both molecules and proteins within a single dec…
Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Survey
Qizhi Pei, Zhimeng Zhou, Kaiyuan Gao +6
The integration of biomolecular modeling with natural language (BL) has emerged as a promising interdisciplinary area at the intersection of artificial intelligence, chemistry and…
BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associations
Qizhi Pei, Wei Zhang, Jinhua Zhu +5
Recent advancements in biological research leverage the integration of molecules, proteins, and natural language to enhance drug discovery. However, current models exhibit several…
AMOM: Adaptive Masking over Masking for Conditional Masked Language Model
Yisheng Xiao, Ruiyang Xu, Lijun Wu +4
Transformer-based autoregressive (AR) methods have achieved appealing performance for varied sequence-to-sequence generation tasks, e.g., neural machine translation, summarization,…
Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Change
Zhaochen Su, Zecheng Tang, Xinyan Guan +3
Recent research has revealed that neural language models at scale suffer from poor temporal generalization capability, i.e., the language model pre-trained on static data from past…
A Mutually Reinforced Framework for Pretrained Sentence Embeddings
Junhan Yang, Zheng Liu, Shitao Xiao +5
The lack of labeled data is a major obstacle to learning high-quality sentence embeddings. Recently, self-supervised contrastive learning (SCL) is regarded as a promising way to ad…