Database Meets Deep Learning: Challenges and Opportunities

Abstract

Deep learning has recently become very popular on account of its incredible success in many complex data-driven applications, such as image classification and speech recognition. The database community has worked on data-driven applications for many years, and therefore should be playing a lead role in supporting this new wave. However, databases and deep learning are different in terms of both techniques and applications. In this paper, we discuss research problems at the intersection of the two fields. In particular, we discuss possible improvements for deep learning systems from a database perspective, and analyze database applications that may benefit from deep learning techniques.

The first version of this paper has appeared in SIGMOD Record. In this (third) version, we extend it to include the recent developments in this field and references to recent work (especially for section 3.2 and section 4.2)

References in corpus (39)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Sequence to Sequence Learning with Neural Networks
• ADADELTA: An Adaptive Learning Rate Method
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• Theano: new features and speed improvements
• Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
• Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning
• TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
• Revisiting Distributed Synchronous SGD
• Training Deep Nets with Sublinear Memory Cost
• Grammar as a Foreign Language
• Deep Image: Scaling up Image Recognition
• SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learning
• Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes
• The PGM-index: a multicriteria, compressed and learned approach to data indexing
• Improving Text-to-SQL Evaluation Methodology
• Deep Reinforcement Learning for Join Order Enumeration
• Device Placement Optimization with Reinforcement Learning
• Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
• Deep Unsupervised Cardinality Estimation
• FITing-Tree: A Data-aware Index Structure
• Staleness-aware Async-SGD for Distributed Deep Learning
• Communication Compression for Decentralized Training
• Deep Learning on FPGAs: Past, Present, and Future
• DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-Pass Error-Compensated Compression
• TrafficPredict: Trajectory Prediction for Heterogeneous Traffic-Agents
• Yet Another Accelerated SGD: ResNet-50 Training on ImageNet in 74.7 seconds
• Towards a Hands-Free Query Optimizer through Deep Learning
• Omnivore: An Optimizer for Multi-device Deep Learning on CPUs and GPUs
• Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques and Tools
• Deep Learning for Spatio-Temporal Data Mining: A Survey
• Model Slicing for Supporting Complex Analytics with Elastic Inference Cost and Resource Constraints
• Active Sampler: Light-weight Accelerator for Complex Data Analytics at Scale
• An Encoder-Decoder Framework Translating Natural Language to Database Queries
• An End-to-End Learning-based Cost Estimator
• DeepSPACE: Approximate Geospatial Query Processing with Deep Learning
• Progressive Neural Index Search for Database System

Cited by in corpus (26)

• Neo: A Learned Query Optimizer
• Software Engineering Challenges of Deep Learning
• Plan-Structured Deep Neural Network Models for Query Performance Prediction
• Cross-Domain Image Retrieval with Attention Modeling
• Scalable Deep Learning on Distributed Infrastructures: Challenges, Techniques and Tools
• GitTables: A Large-Scale Corpus of Relational Tables
• ARM-Net: Adaptive Relation Modeling Network for Structured Data
• Spitz: A Verifiable Database System
• Serverless Data Science -- Are We There Yet? A Case Study of Model Serving
• AlphaEvolve: A Learning Framework to Discover Novel Alphas in Quantitative Investment
• An Empirical Analysis of Deep Learning for Cardinality Estimation
• NeurDB: An AI-powered Autonomous Data System
• Efficient Memory Management for GPU-based Deep Learning Systems
• Rafiki: Machine Learning as an Analytics Service System
• Stochastic Gradient Descent on Highly-Parallel Architectures
• PANDA: Facilitating Usable AI Development
• CorDEL: A Contrastive Deep Learning Approach for Entity Linkage
• Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving
• Progressive Neural Index Search for Database System
• Join Query Optimization with Deep Reinforcement Learning Algorithms
• Communication-efficient Decentralized Machine Learning over Heterogeneous Networks
• CARMI: A Cache-Aware Learned Index with a Cost-based Construction Algorithm
• The Case for Deep Query Optimisation
• In-Machine-Learning Database: Reimagining Deep Learning with Old-School SQL
• MLCask: Efficient Management of Component Evolution in Collaborative Data Analytics Pipelines
• LoCEC: Local Community-based Edge Classification in Large Online Social Networks