Investigating the Successes and Failures of BERT for Passage Re-Ranking

Abstract

The bidirectional encoder representations from transformers (BERT) model has recently advanced the state-of-the-art in passage re-ranking. In this paper, we analyze the results produced by a fine-tuned BERT model to better understand the reasons behind such substantial improvements. To this aim, we focus on the MS MARCO passage re-ranking dataset and provide potential reasons for the successes and failures of BERT for retrieval. In more detail, we empirically study a set of hypotheses and provide additional analysis to explain the successful performance of BERT.

References in corpus (3)

• Distributed Representations of Words and Phrases and their Compositionality
• Natural Language Processing (almost) from Scratch
• Passage Re-ranking with BERT

Cited by in corpus (6)

• Pretrained Transformers for Text Ranking: BERT and Beyond
• On Identifiability in Transformers
• Multi-step Entity-centric Information Retrieval for Multi-Hop Question Answering
• A Study of BERT for Non-Factoid Question-Answering under Passage Length Constraints
• Guided Transformer: Leveraging Multiple External Sources for Representation Learning in Conversational Search
• Transformer Based Language Models for Similar Text Retrieval and Ranking