Automated data processing and feature engineering for deep learning and big data applications: a survey

Abstract

Modern approach to artificial intelligence (AI) aims to design algorithms that learn directly from data. This approach has achieved impressive results and has contributed significantly to the progress of AI, particularly in the sphere of supervised deep learning. It has also simplified the design of machine learning systems as the learning process is highly automated. However, not all data processing tasks in conventional deep learning pipelines have been automated. In most cases data has to be manually collected, preprocessed and further extended through data augmentation before they can be effective for training. Recently, special techniques for automating these tasks have emerged. The automation of data processing tasks is driven by the need to utilize large volumes of complex, heterogeneous data for machine learning and big data applications. Today, end-to-end automated data processing systems based on automated machine learning (AutoML) techniques are capable of taking raw data and transforming them into useful features for Big Data tasks by automating all intermediate processing stages. In this work, we present a thorough review of approaches for automating data processing tasks in deep learning pipelines, including automated data preprocessing--e.g., data cleaning, labeling, missing data imputation, and categorical data encoding--as well as data augmentation (including synthetic data generation using generative AI methods) and feature engineering--specifically, automated feature extraction, feature construction and feature selection. In addition to automating specific data processing tasks, we discuss the use of AutoML methods and tools to simultaneously optimize all stages of the machine learning pipeline.

arXiv commentJournal of Information and Intelligence (2024)

References in corpus (56)

• Scikit-learn: Machine Learning in Python
• TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
• mixup: Beyond Empirical Risk Minimization
• LLaMA: Open and Efficient Foundation Language Models
• Improved Regularization of Convolutional Neural Networks with Cutout
• ICLabel: An automated electroencephalographic independent component classifier, dataset, and website
• Hierarchical Text-Conditional Image Generation with CLIP Latents
• Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
• Diffusion Models in Vision: A Survey
• Neural Architecture Search: A Survey
• AutoML: A Survey of the State-of-the-Art
• Proceedings of the 29th International Conference on Machine Learning (ICML-12)
• OpenML: networked science in machine learning
• GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
• DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems
• Data Synthesis based on Generative Adversarial Networks
• Scaling Autoregressive Models for Content-Rich Text-to-Image Generation
• Towards Expert-Level Medical Question Answering with Large Language Models
• Blended Latent Diffusion
• GridMask Data Augmentation
• Encoding high-cardinality string categorical variables
• Distilling Knowledge from Deep Networks with Applications to Healthcare Domain
• MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Data
• FMix: Enhancing Mixed Sample Data Augmentation
• Automatic Labeling to Generate Training Data for Online LiDAR-based Moving Object Segmentation
• DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
• SaliencyMix: A Saliency Guided Data Augmentation Strategy for Better Regularization
• Adversarial AutoAugment
• Review of automated time series forecasting pipelines
• Automated Evolutionary Approach for the Design of Composite Machine Learning Pipelines
• Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias
• One button machine for automating feature engineering in relational databases
• On Embeddings for Numerical Features in Tabular Deep Learning
• BoostClean: Automated Error Detection and Repair for Machine Learning
• DADA: Differentiable Automatic Data Augmentation
• UniformAugment: A Search-free Probabilistic Data Augmentation Approach
• AlphaClean: Automatic Generation of Data Cleaning Pipelines
• Improving Auto-Augment via Augmentation-Wise Weight Sharing
• BinauralGrad: A Two-Stage Conditional Diffusion Probabilistic Model for Binaural Audio Synthesis
• Analyzing and Mitigating Object Hallucination in Large Vision-Language Models
• Efficient End-to-End AutoML via Scalable Search Space Decomposition
• cleanTS: Automated (AutoML) Tool to Clean Univariate Time Series at Microscales
• Machine Learning With Feature Selection Using Principal Component Analysis for Malware Detection: A Case Study
• Denoising Diffusion Probabilistic Models for Robust Image Super-Resolution in the Wild
• DiffPrep: Differentiable Data Preprocessing Pipeline Search for Learning over Tabular Data
• Uncertain Label Correction via Auxiliary Action Unit Graphs for Facial Expression Recognition
• InPars: Data Augmentation for Information Retrieval using Large Language Models
• ZeroShotDataAug: Generating and Augmenting Training Data with ChatGPT
• Multistep Automated Data Labelling Procedure (MADLaP) for Thyroid Nodules on Ultrasound: An Artificial Intelligence Approach for Automating Image Annotation
• Pseudo-Labeled Auto-Curriculum Learning for Semi-Supervised Keypoint Localization
• Color Space Transformation Network
• Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentation
• Face: Fast, Accurate and Context-Aware Audio Annotation and Classification
• AutoDiff: combining Auto-encoder and Diffusion model for tabular data synthesizing
• AutoDES: AutoML Pipeline Generation of Classification with Dynamic Ensemble Strategy Selection
• Missing Data Infill with Automunge

Cited by in corpus (1)

• Mob-based cattle weight gain forecasting using ML models