416 citations · 1.6k across the 85 of their papers we have counts for
7 papers · 2 filters
LEAD: Liberal Feature-based Distillation for Dense Retrieval
Hao Sun, Xiao Liu, Yeyun Gong +6
Knowledge distillation is often used to transfer knowledge from a strong teacher model to a relatively weak student model. Traditional methods include response-based methods and fe…
PROD: Progressive Distillation for Dense Retrieval
Zhenghao Lin, Yeyun Gong, Xiao Liu +8
Knowledge distillation is an effective way to transfer knowledge from a strong teacher to an efficient student model. Ideally, we expect the better the teacher is, the better the s…
SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrieval
Liang Wang, Nan Yang, Xiaolong Huang +5
In this paper, we propose SimLM (Similarity matching with Language Model pre-training), a simple yet effective pre-training method for dense passage retrieval. It employs a simple…
Unsupervised Context Aware Sentence Representation Pretraining for Multi-lingual Dense Retrieval
Ning Wu, Yaobo Liang, Houxing Ren +4
Recent research demonstrates the effectiveness of using pretrained language models (PLM) to improve dense retrieval and multilingual dense retrieval. In this work, we present a sim…
Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation
Shengyao Zhuang, Houxing Ren, Linjun Shou +4
The Differentiable Search Index (DSI) is an emerging paradigm for information retrieval. Unlike traditional retrieval architectures where index and retrieval are two different and…
Negative Sampling for Contrastive Representation Learning: A Review
Lanling Xu, Jianxun Lian, Wayne Xin Zhao +5
The learn-to-compare paradigm of contrastive representation learning (CRL), which compares positive samples with negative ones for representation learning, has achieved great succe…
Transformer-Empowered Content-Aware Collaborative Filtering
Weizhe Lin, Linjun Shou, Ming Gong +4
Knowledge graph (KG) based Collaborative Filtering is an effective approach to personalizing recommendation systems for relatively static domains such as movies and books, by lever…