176 citations · 914 across the 61 of their papers we have counts for
11 papers · 2 filters
Revisiting Popularity and Demographic Biases in Recommender Evaluation and Effectiveness
Nicola Neophytou, Bhaskar Mitra, Catherine Stinson
Recommendation algorithms are susceptible to popularity bias: a tendency to recommend popular items even when they fail to meet user needs. A related issue is that the recommendati…
Exposing Query Identification for Search Transparency
Ruohan Li, Jianxiang Li, Bhaskar Mitra +2
Search systems control the exposure of ranked content to searchers. In many cases, creators value not only the exposure of their content but, moreover, an understanding of the spec…
Intra-Document Cascading: Learning to Select Passages for Neural Document Ranking
Sebastian Hofstätter, Bhaskar Mitra, Hamed Zamani +2
An emerging recipe for achieving state-of-the-art effectiveness in neural document re-ranking involves utilizing large pre-trained language models - e.g., BERT - to evaluate all in…
Not All Relevance Scores are Equal: Efficient Uncertainty and Calibration Modeling for Deep Retrieval Models
Daniel Cohen, Bhaskar Mitra, Oleg Lesota +2
In any ranking system, the retrieval model outputs a single score for a document based on its belief on how relevant it is to a given search query. While retrieval models have cont…
MS MARCO: Benchmarking Ranking Models in the Large-Data Regime
Nick Craswell, Bhaskar Mitra, Emine Yilmaz +2
Evaluation efforts such as TREC, CLEF, NTCIR and FIRE, alongside public leaderboard such as MS MARCO, are intended to encourage research and track our progress, addressing big ques…
Multi-FR: A Multi-objective Optimization Framework for Multi-stakeholder Fairness-aware Recommendation
Haolun Wu, Chen Ma, Bhaskar Mitra +2
Nowadays, most online services are hosted on multi-stakeholder marketplaces, where consumers and producers may have different objectives. Conventional recommendation systems, howev…
TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime
Nick Craswell, Bhaskar Mitra, Emine Yilmaz +3
The TREC Deep Learning (DL) Track studies ad hoc search in the large data regime, meaning that a large set of human-labeled training data is available. Results so far indicate that…
Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence
Bhaskar Mitra, Sebastian Hofstatter, Hamed Zamani +1
The Transformer-Kernel (TK) model has demonstrated strong reranking performance on the TREC Deep Learning benchmark -- and can be considered to be an efficient (but slightly less e…
Significant Improvements over the State of the Art? A Case Study of the MS MARCO Document Ranking Leaderboard
Jimmy Lin, Daniel Campos, Nick Craswell +2
Leaderboards are a ubiquitous part of modern research in applied machine learning. By design, they sort entries into some linear order, where the top-scoring entry is recognized as…
Overview of the TREC 2020 deep learning track
Nick Craswell, Bhaskar Mitra, Emine Yilmaz +1
This is the second year of the TREC Deep Learning Track, with the goal of studying ad hoc ranking in the large training data regime. We again have a document retrieval task and a p…
Tip of the Tongue Known-Item Retrieval: A Case Study in Movie Identification
Jaime Arguello, Adam Ferguson, Emery Fine +3
While current information retrieval systems are effective for known-item retrieval where the searcher provides a precise name or identifier for the item being sought, systems tend…