345 citations
Counting papers with at most 25 authors, so collaboration author lists do not stand in for the people here.
- Microsoft (United States)US17 papers
- Microsoft Research Montréal (Canada)10 papers
- Carnegie Mellon UniversityUS4 papers
- Georgia Institute of TechnologyUS4 papers
- University College LondonGB4 papers
- University of Illinois Urbana-ChampaignUS4 papers
- Bellevue Hospital CenterUS3 papers
- McGill UniversityCA2 papers
- Microsoft Research New York City (United States)2 papers
- Microsoft Research (United Kingdom)GB2 papers
- Orange (France)FR2 papers
- Princeton UniversityUS2 papers
7 papers · 1 filter
Not All Relevance Scores are Equal: Efficient Uncertainty and Calibration Modeling for Deep Retrieval Models
Daniel Cohen, Bhaskar Mitra, Oleg Lesota +2
In any ranking system, the retrieval model outputs a single score for a document based on its belief on how relevant it is to a given search query. While retrieval models have cont…
MS MARCO: Benchmarking Ranking Models in the Large-Data Regime
Nick Craswell, Bhaskar Mitra, Emine Yilmaz +2
Evaluation efforts such as TREC, CLEF, NTCIR and FIRE, alongside public leaderboard such as MS MARCO, are intended to encourage research and track our progress, addressing big ques…
TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime
Nick Craswell, Bhaskar Mitra, Emine Yilmaz +3
The TREC Deep Learning (DL) Track studies ad hoc search in the large data regime, meaning that a large set of human-labeled training data is available. Results so far indicate that…
Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence
Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani +1
The Transformer-Kernel (TK) model has demonstrated strong reranking performance on the TREC Deep Learning benchmark -- and can be considered to be an efficient (but slightly less e…
Significant Improvements over the State of the Art? A Case Study of the MS MARCO Document Ranking Leaderboard
Jimmy Lin, Daniel Campos, Nick Craswell +2
Leaderboards are a ubiquitous part of modern research in applied machine learning. By design, they sort entries into some linear order, where the top-scoring entry is recognized as…
Tip of the Tongue Known-Item Retrieval: A Case Study in Movie Identification
Jaime Arguello, Adam Ferguson, Emery Fine +3
While current information retrieval systems are effective for known-item retrieval where the searcher provides a precise name or identifier for the item being sought, systems tend…
Evaluating Stochastic Rankings with Expected Exposure
Fernando Diaz, Bhaskar Mitra, Michael D. Ekstrand +2
We introduce the concept of \emph{expected exposure} as the average attention ranked items receive from users over repeated samples of the same query. Furthermore, we advocate for…