WTR: A Test Collection for Web Table Retrieval

Abstract

We describe the development, characteristics and availability of a test collection for the task of Web table retrieval, which uses a large-scale Web Table Corpora extracted from the Common Crawl. Since a Web table usually has rich context information such as the page title and surrounding paragraphs, we not only provide relevance judgments of query-table pairs, but also the relevance judgments of query-table context pairs with respect to a query, which are ignored by previous test collections. To facilitate future research with this benchmark, we provide details about how the dataset is pre-processed and also baseline results from both traditional and recently proposed table retrieval methods. Our experimental results show that proper usage of context labels can benefit previous table retrieval methods.

arXiv commentAccepted as a resource paper in SIGIR 2021

References in corpus (9)

• Which Tasks Should Be Learned Together in Multi-task Learning?
• Answering Table Queries on the Web using Column Keywords
• Ad Hoc Table Retrieval using Semantic Similarity
• QAnswer: Towards Question Answering Search over Websites
• Table2Vec: Neural Word and Entity Embeddings for Table Population and Retrieval
• Table Search Using a Deep Contextualized Language Model
• Novel Entity Discovery from Web Tables
• Design Patterns for Fusion-Based Object Retrieval
• HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Data

Cited by in corpus (2)

• StruBERT: Structure-aware BERT for Table Search and Matching
• Simulating Users in Interactive Web Table Retrieval