Introducing LETOR 4.0 Datasets
arXiv:1306.2597
Abstract
LETOR is a package of benchmark data sets for research on LEarning TO Rank, which contains standard features, relevance judgments, data partitioning, evaluation tools, and several baselines. Version 1.0 was released in April 2007. Version 2.0 was released in Dec. 2007. Version 3.0 was released in Dec. 2008. This version, 4.0, was released in July 2009. Very different from previous versions (V3.0 is an update based on V2.0 and V2.0 is an update based on V1.0), LETOR4.0 is a totally new release. It uses the Gov2 web page collection (~25M pages) and two query sets from Million Query track of TREC 2007 and TREC 2008. We call the two query sets MQ2007 and MQ2008 for short. There are about 1700 queries in MQ2007 with labeled documents and about 800 queries in MQ2008 with labeled documents. If you have any questions or suggestions about the datasets, please kindly email us ([email protected]). Our goal is to make the dataset reliable and useful for the community.
Cited by in corpus (64)
- Evaluating Stochastic Rankings with Expected Exposure
- On Application of Learning to Rank for E-Commerce Search
- Information Retrieval: Recent Advances and Beyond
- Revisiting Deep Learning Models for Tabular Data
- TF-Ranking: Scalable TensorFlow Library for Learning-to-Rank
- Declarative Experimentation in Information Retrieval using PyTerrier
- Unifying Online and Counterfactual Learning to Rank
- Towards Feature Selection for Ranking and Classification Exploiting Quantum Annealers
- Off-policy evaluation for slate recommendation
- Posthoc Interpretability of Learning to Rank Models using Secondary Training Data
- Deep Character-Level Click-Through Rate Prediction for Sponsored Search
- ViTOR: Learning to Rank Webpages Based on Visual Features
- Doubly-Robust Estimation for Correcting Position-Bias in Click Feedback for Unbiased Learning to Rank
- Context-Aware Learning to Rank with Self-Attention
- Ranking for Relevance and Display Preferences in Complex Presentation Layouts
- Taking the Counterfactual Online: Efficient and Unbiased Online Evaluation for Ranking
- Distilled Neural Networks for Efficient Learning to Rank
- Balancing Speed and Quality in Online Learning to Rank for Information Retrieval
- PairRank: Online Pairwise Learning to Rank by Divide-and-Conquer
- RankFormer: Listwise Learning-to-Rank Using Listwide Labels
- B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval
- NeuralNDCG: Direct Optimisation of a Ranking Metric via Differentiable Relaxation of Sorting
- The Power of Selecting Key Blocks with Local Pre-ranking for Long Document Information Retrieval
- DeepTileBars: Visualizing Term Distribution for Neural Information Retrieval
- The Archive Query Log: Mining Millions of Search Result Pages of Hundreds of Search Engines from 25 Years of Web Archives
- Probabilistic Permutation Graph Search: Black-Box Optimization for Fairness in Ranking
- Exact Passive-Aggressive Algorithms for Learning to Rank Using Interval Labels
- Mostra: A Flexible Balancing Framework to Trade-off User, Artist and Platform Objectives for Music Sequencing
- Sparse Dueling Bandits
- Modeling Label Ambiguity for Neural List-Wise Learning to Rank
- Learning from User Interactions with Rankings: A Unification of the Field
- Interpretable Learning-to-Rank with Generalized Additive Models
- Learning Context-Dependent Choice Functions
- Pareto-Optimal Fairness-Utility Amortizations in Rankings with a DBN Exposure Model
- Deep Architectures for Learning Context-dependent Ranking Functions
- Deep differentiable forest with sparse attention for the tabular data
- Modeling Document Interactions for Learning to Rank with Regularized Self-Attention
- MergeDTS: A Method for Effective Large-Scale Online Ranker Evaluation
- BanditRank: Learning to Rank Using Contextual Bandits
- A General Framework for Counterfactual Learning-to-Rank
- Toward the Understanding of Deep Text Matching Models for Information Retrieval
- Valid Explanations for Learning to Rank Models
- A Pre-processing Method for Fairness in Ranking
- Attention augmented differentiable forest for tabular data
- Feature Selection and Model Comparison on Microsoft Learning-to-Rank Data Sets
- A Domain Generalization Perspective on Listwise Context Modeling
- Control Variates for Slate Off-Policy Evaluation
- Metric-agnostic Learning-to-Rank via Boosting and Rank Approximation
- SERank: Optimize Sequencewise Learning to Rank Using Squeeze-and-Excitation Network
- Learning Representations for Axis-Aligned Decision Forests through Input Perturbation
- Counterfactual Learning to Rank using Heterogeneous Treatment Effect Estimation
- Multi-objective Ranking via Constrained Optimization
- A Fast Sampling Gradient Tree Boosting Framework
- ARSM Gradient Estimator for Supervised Learning to Rank
- Computationally Efficient Optimization of Plackett-Luce Ranking Models for Relevance and Fairness
- Dueling Bandits with Qualitative Feedback
- Pessimistic Off-Policy Optimization for Learning to Rank
- Optimizing Ranking Models in an Online Setting
- Conditional Sequential Slate Optimization
- Statistical Consequences of Dueling Bandits
- Booster: An Accelerator for Gradient Boosting Decision Trees
- Generalized Reinforcement Meta Learning for Few-Shot Optimization
- Monotone Retargeting for Unsupervised Rank Aggregation with Object Features
- Distilling Interpretable Models into Human-Readable Code