416 citations · 1.6k across the 85 of their papers we have counts for
9 papers · 2 filters
MASTER: Multi-task Pre-trained Bottlenecked Masked Autoencoders are Better Dense Retrievers
Kun Zhou, Xiao Liu, Yeyun Gong +4
Pre-trained Transformers (\eg BERT) have been commonly used in existing dense retrieval methods for parameter initialization, and recent studies are exploring more effective pre-tr…
Soft-Labeled Contrastive Pre-training for Function-level Code Representation
Xiaonan Li, Daya Guo, Yeyun Gong +6
Code contrastive pre-training has recently achieved significant progress on code-related tasks. In this paper, we present \textbf{SCodeR}, a \textbf{S}oft-labeled contrastive pre-t…
Mixed-modality Representation Learning and Pre-training for Joint Table-and-Text Retrieval in OpenQA
Junjie Huang, Wanjun Zhong, Qian Liu +3
Retrieving evidences from tabular and textual resources is essential for open-domain question answering (OpenQA), which provides more comprehensive information. However, training a…
Rethinking the Value of Gazetteer in Chinese Named Entity Recognition
Qianglong Chen, Xiangji Zeng, Jiangang Zhu +4
Gazetteer is widely used in Chinese named entity recognition (NER) to enhance span boundary detection and type classification. However, to further understand the generalizability a…
Label-aware Multi-level Contrastive Learning for Cross-lingual Spoken Language Understanding
Shining Liang, Linjun Shou, Jian Pei +4
Despite the great success of spoken language understanding (SLU) in high-resource languages, it remains challenging in low-resource languages mainly due to the lack of labeled trai…
Bridging the Gap between Language Models and Cross-Lingual Sequence Labeling
Nuo Chen, Linjun Shou, Ming Gong +2
Large-scale cross-lingual pre-trained language models (xPLMs) have shown effectiveness in cross-lingual sequence labeling tasks (xSL), such as cross-lingual machine reading compreh…
Multi-View Document Representation Learning for Open-Domain Dense Retrieval
Shunyu Zhang, Yaobo Liang, Ming Gong +2
Dense retrieval has achieved impressive advances in first-stage retrieval from a large-scale document collection, which is built on bi-encoder architecture to produce single vector…
PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddings
Qiyu Wu, Chongyang Tao, Tao Shen +3
Learning sentence embeddings in an unsupervised manner is fundamental in natural language processing. Recent common practice is to couple pre-trained language models with unsupervi…
CodeRetriever: Unimodal and Bimodal Contrastive Learning for Code Search
Xiaonan Li, Yeyun Gong, Yelong Shen +7
In this paper, we propose the CodeRetriever model, which learns the function-level code semantic representations through large-scale code-text contrastive pre-training. We adopt tw…