Dataset search: a survey
arXiv:1901.00735 · doi:10.1007/s00778-019-00564-x
Abstract
Generating value from data requires the ability to find, access and make sense of datasets. There are many efforts underway to encourage data sharing and reuse, from scientific publishers asking authors to submit data alongside manuscripts to data marketplaces, open data portals and data communities. Google recently beta released a search service for datasets, which allows users to discover data stored in various online repositories via keyword queries. These developments foreshadow an emerging research field around dataset search or retrieval that broadly encompasses frameworks, methods and tools that help match a user data need against a collection of datasets. Here, we survey the state of the art of research and commercial systems in dataset retrieval. We identify what makes dataset search a research field in its own right, with unique challenges and methods and highlight open problems. We look at approaches and implementations from related areas dataset search is drawing upon, including information retrieval, databases, entity-centric and tabular search in order to identify possible paths to resolve these open problems as well as immediate next steps that will take the field forward.
arXiv comment20 pages, 153 references
References in corpus (9)
- Apache Calcite: A Foundational Framework for Optimized Query Processing Over Heterogeneous Data Sources
- A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration Perspective
- Answering Table Queries on the Web using Column Keywords
- Ad Hoc Table Retrieval using Semantic Similarity
- Assessing and Remedying Coverage for a Given Dataset
- Neural Vector Spaces for Unsupervised Information Retrieval
- EntiTables: Smart Assistance for Entity-Focused Tables
- On-the-fly Table Generation
- Cuttlefish: A Lightweight Primitive for Adaptive Query Processing