125 citations · 326 across the 20 of their papers we have counts for
20 papers
xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein
Bo Chen, Xingyi Cheng, Pan Li +12
Protein language models have shown remarkable success in learning biological information from protein sequences. However, most existing models are limited by either autoencoding or…
xTrimoGene: An Efficient and Scalable Representation Learner for Single-Cell RNA-Seq Data
Jing Gong, Minsheng Hao, Xingyi Cheng +6
Advances in high-throughput sequencing technology have led to significant progress in measuring gene expressions at the single-cell level. The amount of publicly available single-c…
Won't Get Fooled Again: Answering Questions with False Premises
Shengding Hu, Yifan Luo, Huadong Wang +3
Pre-trained language models (PLMs) have shown unprecedented potential in various fields, especially as the backbones for question-answering (QA) systems. However, they tend to be e…
Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations
Lifan Yuan, Yangyi Chen, Ganqu Cui +6
This paper reexamines the research on out-of-distribution (OOD) robustness in the field of NLP. We find that the distribution shift settings in previous studies commonly lack adequ…
Generation-Guided Multi-Level Unified Network for Video Grounding
Xing Cheng, Xiangyu Wu, Dong Shen +2
Video grounding aims to locate the timestamps best matching the query description within an untrimmed video. Prevalent methods can be divided into moment-level and clip-level frame…
SimViT: Exploring a Simple Vision Transformer with sliding windows
Gang Li, Di Xu, Xing Cheng +2
Although vision Transformers have achieved excellent performance as backbone models in many vision tasks, most of them intend to capture global relations of all tokens in an image…
K-AID: Enhancing Pre-trained Language Models with Domain Knowledge for Question Answering
Fu Sun, Feng-Lin Li, Ruize Wang +3
Knowledge enhanced pre-trained language models (K-PLMs) are shown to be effective for many public tasks in the literature but few of them have been successfully applied in practice…
Improving Video-Text Retrieval by Multi-Stream Corpus Alignment and Dual Softmax Loss
Xing Cheng, Hezheng Lin, Xiangyu Wu +2
Employing large-scale pre-trained model CLIP to conduct video-text retrieval task (VTR) has become a new trend, which exceeds previous VTR methods. Though, due to the heterogeneity…
MlTr: Multi-label Classification with Transformer
Xing Cheng, Hezheng Lin, Xiangyu Wu +5
The task of multi-label image classification is to recognize all the object labels presented in an image. Though advancing for years, small objects, similar objects and objects wit…
CAT: Cross Attention in Vision Transformer
Hezheng Lin, Xing Cheng, Xiangyu Wu +5
Since Transformer has found widespread use in NLP, the potential of Transformer in CV has been realized and has inspired many new approaches. However, the computation required for…
Dual-View Distilled BERT for Sentence Embedding
Xingyi Cheng
Recently, BERT realized significant progress for sentence matching via word-level cross sentence attention. However, the performance significantly drops when using siamese BERT-net…
Question Directed Graph Attention Network for Numerical Reasoning over Text
Kunlong Chen, Weidi Xu, Xingyi Cheng +6
Numerical reasoning over texts, such as addition, subtraction, sorting and counting, is a challenging machine reading comprehension task, since it requires both natural language un…
A Workflow for Exploring Ligand Dissociation from a Macromolecule: Efficient Random Acceleration Molecular Dynamics Simulation and Interaction Fingerprints Analysis of Ligand Trajectories
Daria B. Kokha, Bernd Doser, Stefan Richter +3
The dissociation of ligands from proteins and other biomacromolecules occurs over a wide range of timescales. For most pharmaceutically relevant inhibitors, these timescales are fa…
SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check
Xingyi Cheng, Weidi Xu, Kunlong Chen +5
Chinese Spelling Check (CSC) is a task to detect and correct spelling errors in Chinese natural language. Existing methods have made attempts to incorporate the similarity knowledg…
BERT-Based Multi-Head Selection for Joint Entity-Relation Extraction
Weipeng Huang, Xingyi Cheng, Taifeng Wang +1
In this paper, we report our method for the Information Extraction task in 2019 Language and Intelligence Challenge. We incorporate BERT into the multi-head selection framework for…
Symmetric Regularization based BERT for Pair-wise Semantic Reasoning
Weidi Xu, Xingyi Cheng, Kunlong Chen +7
The ability of semantic reasoning over the sentence pair is essential for many natural language understanding tasks, e.g., natural language inference and machine reading comprehens…
An end-to-end Neural Network Framework for Text Clustering
Jie Zhou, Xingyi Cheng, Jinchao Zhang
The unsupervised text clustering is one of the major tasks in natural language processing (NLP) and remains a difficult and complex problem. Conventional \mbox{methods} generally t…
Toward Fast and Accurate Neural Chinese Word Segmentation with Multi-Criteria Learning
Weipeng Huang, Xingyi Cheng, Kunlong Chen +2
The ambiguous annotation criteria lead to divergence of Chinese Word Segmentation (CWS) datasets in various granularities. Multi-criteria Chinese word segmentation aims to capture…
Variational Semi-supervised Aspect-term Sentiment Analysis via Transformer
Xingyi Cheng, Weidi Xu, Taifeng Wang +1
Aspect-term sentiment analysis (ATSA) is a longstanding challenge in natural language understanding. It requires fine-grained semantical reasoning about a target entity appeared in…
DeepTransport: Learning Spatial-Temporal Dependency for Traffic Condition Forecasting
Xingyi Cheng, Ruiqing Zhang, Jie Zhou +1
Predicting traffic conditions has been recently explored as a way to relieve traffic congestion. Several pioneering approaches have been proposed based on traffic observations of t…