306 citations · 665 across the 10 of their papers we have counts for
11 papers
Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Change
Zhaochen Su, Zecheng Tang, Xinyan Guan +3
Recent research has revealed that neural language models at scale suffer from poor temporal generalization capability, i.e., the language model pre-trained on static data from past…
A Mutually Reinforced Framework for Pretrained Sentence Embeddings
Junhan Yang, Zheng Liu, Shitao Xiao +5
The lack of labeled data is a major obstacle to learning high-quality sentence embeddings. Recently, self-supervised contrastive learning (SCL) is regarded as a promising way to ad…
How to Leverage Multimodal EHR Data for Better Medical Predictions?
Bo Yang, Lijun Wu
Healthcare is becoming a more and more important research topic recently. With the growing data in the healthcare domain, it offers a great opportunity for deep learning to improve…
Pre-training Co-evolutionary Protein Representation via A Pairwise Masked Language Model
Liang He, Shizhuo Zhang, Lijun Wu +10
Understanding protein sequences is vital and urgent for biology, healthcare, and medicine. Labeling approaches are expensive yet time-consuming, while the amount of unlabeled data…
Discovering Drug-Target Interaction Knowledge from Biomedical Literature
Yutai Hou, Yingce Xia, Lijun Wu +6
The Interaction between Drugs and Targets (DTI) in human body plays a crucial role in biomedical science and applications. As millions of papers come out every year in the biomedic…
R-Drop: Regularized Dropout for Neural Networks
Xiaobo Liang, Lijun Wu, Juntao Li +6
Dropout is a powerful and widely used technique to regularize the training of deep neural networks. In this paper, we introduce a simple regularization strategy upon dropout in mod…