73 citations · 522 across the 47 of their papers we have counts for
34 papers · 1 filter
Revisiting N2DCG: An Empirically Grounded Reformulation of Carousel Recommendation Evaluation
Jingwei Kang, Santiago de Leon-Martinez, Maarten de Rijke +1
Carousel interfaces have been widely used in video and music streaming services, yet it remains unclear how to properly evaluate recommender systems in these two-dimensional layout…
An Epistemic Position-Based Click Model: From Interactions to Epistemic Distributions of Relevance and Bias
Oscar Rolando Ramirez Milian, Harrie Oosterhuis
User interactions with rankings are affected by both items' relevances and display positions. Accordingly, click probabilities are often modeled as a product of relevance and posit…
Following the Eye-Tracking Evidence: Established Web-Search Assumptions Fail in Carousel Interfaces
Jingwei Kang, Maarten de Rijke, Harrie Oosterhuis
Carousel interfaces have been the de-facto standard for streaming media services for over a decade. Yet, there has been very little research into user behavior with such interfaces…
A Non-Parametric Choice Model That Learns How Users Choose Between Recommended Options
Thorsten Krause, Harrie Oosterhuis
Choice models predict which items users choose from presented options. In recommendation settings, they can infer user preferences while countering exposure bias. In contrast with…
Harnessing Pairwise Ranking Prompting Through Sample-Efficient Ranking Distillation
Junru Wu, Le Yan, Zhen Qin +6
While Pairwise Ranking Prompting (PRP) with Large Language Models (LLMs) is one of the most effective zero-shot document ranking methods, it has a quadratic computational complexit…
Rethinking Click Models in Light of Carousel Interfaces: Theory-Based Categorization and Design of Click Models
Jingwei Kang, Maarten de Rijke, Santiago de Leon-Martinez +1
Click models are a well-established for modeling user interactions with web interfaces. Previous work has mainly focused on traditional single-list web search settings; this includ…
Learning to Rank with Variable Result Presentation Lengths
Norman Knyazev, Harrie Oosterhuis
Learning to Rank (LTR) methods generally assume that each document in a top-K ranking is presented in an equal format. However, previous work has shown that users' perceptions of r…
RecGaze: The First Eye Tracking and User Interaction Dataset for Carousel Interfaces
Santiago de Leon-Martinez, Jingwei Kang, Robert Moro +4
Carousel interfaces are widely used in e-commerce and streaming services, but little research has been devoted to them. Previous studies of interfaces for presenting search and rec…
Adaptive Orchestration of Modular Generative Information Access Systems
Mohanna Hoveyda, Harrie Oosterhuis, Arjen P. de Vries +2
Advancements in large language models (LLMs) have driven the emergence of complex new systems to provide access to information, that we will collectively refer to as modular genera…
Optimizing Compound Retrieval Systems
Harrie Oosterhuis, Rolf Jagerman, Zhen Qin +1
Modern retrieval systems do not rely on a single ranking model to construct their rankings. Instead, they generally take a cascading approach where a sequence of ranking models are…
Reliable Confidence Intervals for Information Retrieval Evaluation Using Generative A.I
Harrie Oosterhuis, Rolf Jagerman, Zhen Qin +2
The traditional evaluation of information retrieval (IR) systems is generally very costly as it requires manual relevance annotation from human experts. Recent advancements in gene…
Is Interpretable Machine Learning Effective at Feature Selection for Neural Learning-to-Rank?
Lijun Lyu, Nirmal Roy, Harrie Oosterhuis +1
Neural ranking models have become increasingly popular for real-world search and recommendation systems in recent years. Unlike their tree-based counterparts, neural models are muc…
A First Look at Selection Bias in Preference Elicitation for Recommendation
Shashank Gupta, Harrie Oosterhuis, Maarten de Rijke
Preference elicitation explicitly asks users what kind of recommendations they would like to receive. It is a popular technique for conversational recommender systems to deal with…
Going Beyond Popularity and Positivity Bias: Correcting for Multifactorial Bias in Recommender Systems
Jin Huang, Harrie Oosterhuis, Masoud Mansoury +2
Two typical forms of bias in user interaction data with recommender systems (RSs) are popularity bias and positivity bias, which manifest themselves as the over-representation of i…
Consolidating Ranking and Relevance Predictions of Large Language Models through Post-Processing
Le Yan, Zhen Qin, Honglei Zhuang +4
The powerful generative abilities of large language models (LLMs) show potential in generating relevance labels for search applications. Previous work has found that directly askin…
A Lightweight Method for Modeling Confidence in Recommendations with Learned Beta Distributions
Norman Knyazev, Harrie Oosterhuis
Most Recommender Systems (RecSys) do not provide an indication of confidence in their decisions. Therefore, they do not distinguish between recommendations of which they are certai…
Recent Advances in the Foundations and Applications of Unbiased Learning to Rank
Shashank Gupta, Philipp Hager, Jin Huang +2
Since its inception, the field of unbiased learning to rank (ULTR) has remained very active and has seen several impactful advancements in recent years. This tutorial provides both…
Safe Deployment for Counterfactual Learning to Rank with Exposure-Based Risk Minimization
Shashank Gupta, Harrie Oosterhuis, Maarten de Rijke
Counterfactual learning to rank (CLTR) relies on exposure-based inverse propensity scoring (IPS), a LTR-specific adaptation of IPS to correct for position bias. While IPS can provi…
The Bandwagon Effect: Not Just Another Bias
Norman Knyazev, Harrie Oosterhuis
Optimizing recommender systems based on user interaction data is mainly seen as a problem of dealing with selection bias, where most existing work assumes that interactions from di…
Reaching the End of Unbiasedness: Uncovering Implicit Limitations of Click-Based Learning to Rank
Harrie Oosterhuis
Click-based learning to rank (LTR) tackles the mismatch between click frequencies on items and their actual relevance. The approach of previous work has been to assume a model of c…
State Encoders in Reinforcement Learning for Recommendation: A Reproducibility Study
Jin Huang, Harrie Oosterhuis, Bunyamin Cetinkaya +2
Methods for reinforcement learning for recommendation (RL4Rec) are increasingly receiving attention as they can quickly adapt to user feedback. A typical RL4Rec framework consists…
It Is Different When Items Are Older: Debiasing Recommendations When Selection Bias and User Preferences Are Dynamic
Jin Huang, Harrie Oosterhuis, Maarten de Rijke
User interactions with recommender systems (RSs) are affected by user selection bias, e.g., users are more likely to rate popular items (popularity bias) or items that they expect…
Computationally Efficient Optimization of Plackett-Luce Ranking Models for Relevance and Fairness
Harrie Oosterhuis
Recent work has proposed stochastic Plackett-Luce (PL) ranking models as a robust choice for optimizing relevance and fairness metrics. Unlike their deterministic counterparts that…
Learning from User Interactions with Rankings: A Unification of the Field
Harrie Oosterhuis
Ranking systems form the basis for online search engines and recommendation services. They process large collections of items, for instance web pages or e-commerce products, and pr…