fastai: A Layered API for Deep Learning

Abstract

fastai is a deep learning library which provides practitioners with high-level components that can quickly and easily provide state-of-the-art results in standard deep learning domains, and provides researchers with low-level components that can be mixed and matched to build new approaches. It aims to do both things without substantial compromises in ease of use, flexibility, or performance. This is possible thanks to a carefully layered architecture, which expresses common underlying patterns of many deep learning and data processing techniques in terms of decoupled abstractions. These abstractions can be expressed concisely and clearly by leveraging the dynamism of the underlying Python language and the flexibility of the PyTorch library. fastai includes: a new type dispatch system for Python along with a semantic type hierarchy for tensors; a GPU-optimized computer vision library which can be extended in pure Python; an optimizer which refactors out the common functionality of modern optimizers into two basic pieces, allowing optimization algorithms to be implemented in 4-5 lines of code; a novel 2-way callback system that can access any part of the data, model, or optimizer and change it at any point during training; a new data block API; and much more. We have used this library to successfully create a complete deep learning course, which we were able to write more quickly than using previous approaches, and the code was more clear. The library is already in wide use in research, industry, and teaching. NB: This paper covers fastai v2, which is currently in pre-release at http://dev.fast.ai/

References in corpus (18)

• Scikit-learn: Machine Learning in Python
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• mixup: Beyond Empirical Risk Minimization
• Caffe: Convolutional Architecture for Fast Feature Embedding
• HuggingFace's Transformers: State-of-the-art Natural Language Processing
• Squeeze-and-Excitation Networks
• Densely Connected Convolutional Networks
• Mixed Precision Training
• A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay
• Regularizing and Optimizing LSTM Language Models
• Entity Embeddings of Categorical Variables
• fairseq: A Fast, Extensible Toolkit for Sequence Modeling
• Bag of Tricks for Image Classification with Convolutional Neural Networks
• Artificial Neural Networks Applied to Taxi Destination Prediction
• Hierarchical ResNeXt Models for Breast Cancer Histology Image Classification
• Multiclass Language Identification using Deep Learning on Spectral Images of Audio Signals
• Suggestion Mining from Online Reviews using ULMFiT

Cited by in corpus (72)

• SpeechBrain: A General-Purpose Speech Toolkit
• COVID-ResNet: A Deep Learning Framework for Screening of COVID19 from Radiographs
• Automated Pavement Crack Segmentation Using U-Net-based Convolutional Neural Network
• A survey on computational spectral reconstruction methods from RGB to hyperspectral imaging
• AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data
• Data Augmentation in Natural Language Processing: A Novel Text Generation Approach for Long and Short Text Classifiers
• Comparing Different Deep Learning Architectures for Classification of Chest Radiographs
• Blind Backdoors in Deep Learning Models
• Segmentation of Satellite Imagery using U-Net Models for Land Cover Classification
• ktrain: A Low-Code Library for Augmented Machine Learning
• Ten Quick Tips for Deep Learning in Biology
• CXR-Net: An Encoder-Decoder-Encoder Multitask Deep Neural Network for Explainable and Accurate Diagnosis of COVID-19 pneumonia with Chest X-ray Images
• One Label, One Billion Faces: Usage and Consistency of Racial Categories in Computer Vision
• Neural Networks Versus Conventional Filters for Inertial-Sensor-based Attitude Estimation
• The Ivory Tower Lost: How College Students Respond Differently than the General Public to the COVID-19 Pandemic
• Pan-tumor CAnine cuTaneous Cancer Histology (CATCH) dataset
• Convolutional Neural Networks for the classification of glitches in gravitational-wave data streams
• Automatic Large Scale Detection of Red Palm Weevil Infestation using Aerial and Street View Images
• Connecting optical morphology, environment, and HI mass fraction for low-redshift galaxies using deep learning
• Blending gradient boosted trees and neural networks for point and probabilistic forecasting of hierarchical time series
• E-Scooter Rider Detection and Classification in Dense Urban Environments
• Learning minimal representations of stochastic processes with variational autoencoders
• PatchBackdoor: Backdoor Attack against Deep Neural Networks without Model Modification
• MXR-U-Nets for Real Time Hyperspectral Reconstruction
• Predicting brain-age from raw T 1 -weighted Magnetic Resonance Imaging data using 3D Convolutional Neural Networks
• Online Algorithms for Hierarchical Inference in Deep Learning applications at the Edge
• EnsembleCI: Ensemble Learning for Carbon Intensity Forecasting
• indicnlp@kgp at DravidianLangTech-EACL2021: Offensive Language Identification in Dravidian Languages
• Improving the HardNet Descriptor
• Predicting galaxy spectra from images with hybrid convolutional neural networks
• Towards Democratizing AI: A Comparative Analysis of AI as a Service Platforms and the Open Space for Machine Learning Approach
• Image segmentation of treated and untreated tumor spheroids by Fully Convolutional Networks
• Aerial Imagery Pixel-level Segmentation
• Imprinto: Enhancing Infrared Inkjet Watermarking for Human and Machine Perception
• Data Efficient Stagewise Knowledge Distillation
• ImageNet-21K Pretraining for the Masses
• Muddling Label Regularization: Deep Learning for Tabular Datasets
• Detecting Hostile Posts using Relational Graph Convolutional Network
• SHREC 2025: Protein surface shape retrieval including electrostatic potential
• Is Attention always needed? A Case Study on Language Identification from Speech
• PyTorch Tabular: A Framework for Deep Learning with Tabular Data
• Automated Source Code Generation and Auto-completion Using Deep Learning: Comparing and Discussing Current Language-Model-Related Approaches
• How Much Data is Enough? Optimization of Data Collection for Artifact Detection in EEG Recordings
• Gated recurrent units and temporal convolutional network for multilabel classification
• FastSHAP: Real-Time Shapley Value Estimation
• Crash Severity Risk Modeling Strategies under Data Imbalance
• Insights on Galaxy Evolution from Interpretable Sparse Feature Networks
• RIANN -- A Robust Neural Network Outperforms Attitude Estimation Filters
• Predicting Account Receivables with Machine Learning
• 2kenize: Tying Subword Sequences for Chinese Script Conversion
• Exploratory Data Science on Supercomputers for Quantum Mechanical Calculations
• Deep-learning in the bioimaging wild: Handling ambiguous data with deepflash2
• Toward Improving Coherence and Diversity of Slogan Generation
• Cryo-forum: A framework for orientation recovery with uncertainty measure with the application in cryo-EM image analysis
• Seeing poverty from space, how much can it be tuned?
• Comparative Analysis of Deep Learning Algorithms for Classification of COVID-19 X-Ray Images
• AI-driven Orchestration at Scale: Estimating Service Metrics on National-Wide Testbeds
• Probing Network Decisions: Capturing Uncertainties and Unveiling Vulnerabilities Without Label Information
• Cross-lingual Transfer for Text Classification with Dictionary-based Heterogeneous Graph
• MapReader: A Computer Vision Pipeline for the Semantic Exploration of Maps at Scale
• Deep learning and whole-brain networks for biomarker discovery: modeling the dynamics of brain fluctuations in resting-state and cognitive tasks
• Deep Medical Image Analysis with Representation Learning and Neuromorphic Computing
• Mining for Potent Inhibitors through Artificial Intelligence and Physics: A Unified Methodology for Ligand Based and Structure Based Drug Design
• Multi-concept adversarial attacks
• A Practical Framework for ROI Detection in Medical Images -- a case study for hip detection in anteroposterior pelvic radiographs
• TensorX: Extensible API for Neural Network Model Design and Deployment
• Full waveform inversion with random shot selection using adaptive gradient descent
• Can neural networks count digit frequency?
• Induced Feature Selection by Structured Pruning
• Exploiting News Article Structure for Automatic Corpus Generation of Entailment Datasets
• Extending the SAGA Survey (xSAGA) I: Satellite Radial Profiles as a Function of Host Galaxy Properties
• Explicit Gradient Learning