221 citations · 587 across the 23 of their papers we have counts for
13 papers · 1 filter
Tevatron Meets Megatron: Expert-Parallel LLM Reranker Training on an Academic Budget
Zhichao Xu, Xueguang Ma, Shengyao Zhuang +5
Modern reranking recipes---billion-scale cross-encoders, mixture-of-experts (MoE) backbones, and distillation against strong teachers---have outpaced the training infrastructure av…
Tevatron 2.0: Unified Document Retrieval Toolkit across Scale, Language, and Modality
Xueguang Ma, Luyu Gao, Shengyao Zhuang +3
Recent advancements in large language models (LLMs) have driven interest in billion-scale retrieval models with strong generalization across retrieval tasks and languages. Addition…
Repository-level Code Search with Neural Retrieval Methods
Siddharth Gandhi, Luyu Gao, Jamie Callan
This paper presents a multi-stage reranking system for repository-level code search, which leverages the vastly available commit histories of large open-source repositories to aid…
DataFinder: Scientific Dataset Recommendation from Natural Language Descriptions
Vijay Viswanathan, Luyu Gao, Tongshuang Wu +2
Modern machine learning relies on datasets to develop and validate research ideas. Given the growth of publicly available data, finding the right dataset to use is increasingly dif…
Precise Zero-Shot Dense Retrieval without Relevance Labels
Luyu Gao, Xueguang Ma, Jimmy Lin +1
While dense retrieval has been shown effective and efficient across tasks and languages, it remains difficult to create effective fully zero-shot dense retrieval systems when no re…
Long Document Re-ranking with Modular Re-ranker
Luyu Gao, Jamie Callan
Long document re-ranking has been a challenging problem for neural re-rankers based on deep language models like BERT. Early work breaks the documents into short passage-like chunk…
Tevatron: An Efficient and Flexible Toolkit for Dense Retrieval
Luyu Gao, Xueguang Ma, Jimmy Lin +1
Recent rapid advancements in deep pre-trained language models and the introductions of large datasets have powered research in embedding-based dense retrieval. While several good r…
Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval
Luyu Gao, Jamie Callan
Recent research demonstrates the effectiveness of using fine-tuned language models~(LM) for dense retrieval. However, dense retrievers are hard to train, typically requiring heavil…
COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List
Luyu Gao, Zhuyun Dai, Jamie Callan
Classical information retrieval systems such as BM25 rely on exact lexical match and carry out search efficiently with inverted list index. Recent neural IR models shifts towards s…
Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipeline
Luyu Gao, Zhuyun Dai, Jamie Callan
Pre-trained deep language models~(LM) have advanced the state-of-the-art of text retrieval. Rerankers fine-tuned from deep LM estimates candidate relevance based on rich contextual…
Understanding BERT Rankers Under Distillation
Luyu Gao, Zhuyun Dai, Jamie Callan
Deep language models such as BERT pre-trained on large corpus have given a huge performance boost to the state-of-the-art information retrieval ranking systems. Knowledge embedded…
Complementing Lexical Retrieval with Semantic Residual Embedding
Luyu Gao, Zhuyun Dai, Tongfei Chen +3
This paper presents CLEAR, a retrieval model that seeks to complement classical lexical exact-match models such as BM25 with semantic matching signals from a neural embedding match…
Modularized Transfomer-based Ranking Framework
Luyu Gao, Zhuyun Dai, Jamie Callan
Recent innovations in Transformer-based ranking models have advanced the state-of-the-art in information retrieval. However, these Transformers are computationally expensive, and t…