Automated data processing and feature engineering for deep learning and big data applications: a survey

Abstract

Modern approach to artificial intelligence (AI) aims to design algorithms that learn directly from data. This approach has achieved impressive results and has contributed significantly to the progress of AI, particularly in the sphere of supervised deep learning. It has also simplified the design of machine learning systems as the learning process is highly automated. However, not all data processing tasks in conventional deep learning pipelines have been automated. In most cases data has to be manually collected, preprocessed and further extended through data augmentation before they can be effective for training. Recently, special techniques for automating these tasks have emerged. The automation of data processing tasks is driven by the need to utilize large volumes of complex, heterogeneous data for machine learning and big data applications. Today, end-to-end automated data processing systems based on automated machine learning (AutoML) techniques are capable of taking raw data and transforming them into useful features for Big Data tasks by automating all intermediate processing stages. In this work, we present a thorough review of approaches for automating data processing tasks in deep learning pipelines, including automated data preprocessing--e.g., data cleaning, labeling, missing data imputation, and categorical data encoding--as well as data augmentation (including synthetic data generation using generative AI methods) and feature engineering--specifically, automated feature extraction, feature construction and feature selection. In addition to automating specific data processing tasks, we discuss the use of AutoML methods and tools to simultaneously optimize all stages of the machine learning pipeline.

Journal of Information and Intelligence (2024)

References in corpus (23)

• LLaMA: Open and Efficient Foundation Language Models
• Improved Regularization of Convolutional Neural Networks with Cutout
• Hierarchical Text-Conditional Image Generation with CLIP Latents
• Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
• Proceedings of the 29th International Conference on Machine Learning (ICML-12)
• OpenML: networked science in machine learning
• Towards Expert-Level Medical Question Answering with Large Language Models
• Automatic Labeling to Generate Training Data for Online LiDAR-based Moving Object Segmentation
• Adversarial AutoAugment
• Automated Evolutionary Approach for the Design of Composite Machine Learning Pipelines
• One button machine for automating feature engineering in relational databases
• BoostClean: Automated Error Detection and Repair for Machine Learning
• AlphaClean: Automatic Generation of Data Cleaning Pipelines
• Machine Learning With Feature Selection Using Principal Component Analysis for Malware Detection: A Case Study
• Denoising Diffusion Probabilistic Models for Robust Image Super-Resolution in the Wild
• DiffPrep: Differentiable Data Preprocessing Pipeline Search for Learning over Tabular Data
• InPars: Data Augmentation for Information Retrieval using Large Language Models
• ZeroShotDataAug: Generating and Augmenting Training Data with ChatGPT
• Pseudo-Labeled Auto-Curriculum Learning for Semi-Supervised Keypoint Localization
• Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentation
• AutoDiff: combining Auto-encoder and Diffusion model for tabular data synthesizing
• Face: Fast, Accurate and Context-Aware Audio Annotation and Classification
• Missing Data Infill with Automunge