Table Search Using a Deep Contextualized Language Model

Abstract

Pretrained contextualized language models such as BERT have achieved impressive results on various natural language processing benchmarks. Benefiting from multiple pretraining tasks and large scale training corpora, pretrained models can capture complex syntactic word relations. In this paper, we use the deep contextualized language model BERT for the task of ad hoc table retrieval. We investigate how to encode table content considering the table structure and input length limit of BERT. We also propose an approach that incorporates features from prior literature on table retrieval and jointly trains them with BERT. In experiments on public datasets, we show that our best approach can outperform the previous state-of-the-art method and BERT baselines with a large margin under different evaluation metrics.

Accepted at SIGIR 2020 (Long)

References in corpus (5)

• End-to-End Open-Domain Question Answering with BERTserini
• Deeper Text Understanding for IR with Contextual Neural Language Modeling
• CEDR: Contextualized Embeddings for Document Ranking
• Ad Hoc Table Retrieval using Semantic Similarity
• Table2Vec: Neural Word and Entity Embeddings for Table Population and Retrieval

Cited by in corpus (5)

• Neural ranking models for document retrieval
• StruBERT: Structure-aware BERT for Table Search and Matching
• Retrieving Complex Tables with Multi-Granular Graph Representation Learning
• Semantic Table Retrieval using Keyword and Table Queries
• WTR: A Test Collection for Web Table Retrieval