A Survey on Data Collection for Machine Learning: a Big Data -- AI Integration Perspective

Abstract

Data collection is a major bottleneck in machine learning and an active research topic in multiple communities. There are largely two reasons data collection has recently become a critical issue. First, as machine learning is becoming more widely-used, we are seeing new applications that do not necessarily have enough labeled data. Second, unlike traditional machine learning, deep learning techniques automatically generate features, which saves feature engineering costs, but in return may require larger amounts of labeled data. Interestingly, recent research in data collection comes not only from the machine learning, natural language, and computer vision communities, but also from the data management community due to the importance of handling large amounts of data. In this survey, we perform a comprehensive study of data collection from a data management point of view. Data collection largely consists of data acquisition, data labeling, and improvement of existing data or models. We provide a research landscape of these operations, provide guidelines on which technique to use when, and identify interesting research challenges. The integration of machine learning and data management for data collection is part of a larger trend of Big data and Artificial Intelligence (AI) integration and opens many opportunities for new research.

arXiv comment20 pages

References in corpus (16)

• Scikit-learn: Machine Learning in Python
• SMOTE: Synthetic Minority Over-sampling Technique
• Distributed Representations of Words and Phrases and their Compositionality
• Distributed Representations of Sentences and Documents
• How transferable are features in deep neural networks?
• NIPS 2016 Tutorial: Generative Adversarial Networks
• Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition
• Snorkel: Rapid Training Data Creation with Weak Supervision
• AutoAugment: Learning Augmentation Policies from Data
• Data Synthesis based on Generative Adversarial Networks
• Web Data Extraction, Applications and Techniques: A Survey
• Generating Multi-label Discrete Patient Records using Generative Adversarial Networks
• Synthesizing Tabular Data using Generative Adversarial Networks
• Learning the Structure of Generative Models without Labeled Data
• BoostClean: Automated Error Detection and Repair for Machine Learning
• Error Rate Bounds in Crowdsourcing Models

Cited by in corpus (11)

• Assuring the Machine Learning Lifecycle: Desiderata, Methods, and Challenges
• Arachne: Search Based Repair of Deep Neural Networks
• Leveraging Siamese Networks for One-Shot Intrusion Detection Model
• Machine Learning Software Engineering in Practice: An Industrial Case Study
• The Lean Data Scientist: Recent Advances towards Overcoming the Data Bottleneck
• Dataset search: a survey
• Automatic Generation of Machine Learning Synthetic Data Using ROS
• Batchwise Probabilistic Incremental Data Cleaning
• Finding the Ground-Truth from Multiple Labellers: Why Parameters of the Task Matter
• Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows
• Self-supervised Deep Learning for Reading Activity Classification