TorchDyn: A Neural Differential Equations Library
arXiv:2009.09346
Abstract
Continuous-depth learning has recently emerged as a novel perspective on deep learning, improving performance in tasks related to dynamical systems and density estimation. Core to these approaches is the neural differential equation, whose forward passes are the solutions of an initial value problem parametrized by a neural network. Unlocking the full potential of continuous-depth models requires a different set of software tools, due to peculiar differences compared to standard discrete neural networks, e.g inference must be carried out via numerical solvers. We introduce TorchDyn, a PyTorch library dedicated to continuous-depth learning, designed to elevate neural differential equations to be as accessible as regular plug-and-play deep learning primitives. This objective is achieved by identifying and subdividing different variants into common essential components, which can be combined and freely repurposed to obtain complex compositional architectures. TorchDyn further offers step-by-step tutorials and benchmarks designed to guide researchers and contributors.
References in corpus (14)
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Scalable Gradients for Stochastic Differential Equations
- FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
- How to train your neural ODE: the world of Jacobian and kinetic regularization
- DiffEqFlux.jl - A Julia Library for Neural Differential Equations
- Dissecting Neural ODEs
- Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning
- Lagrangian Neural Networks
- Graph Neural Ordinary Differential Equations
- Hamiltonian Generative Networks
- Turbulence forecasting via Neural ODE
- Hypersolvers: Toward Fast Continuous-Depth Models
- Neural Ordinary Differential Equations for Semantic Segmentation of Individual Colon Glands
- Stable Neural Flows
Cited by in corpus (7)
- Dissecting Neural ODEs
- Accelerating Neural ODEs Using Model Order Reduction
- Hypersolvers: Toward Fast Continuous-Depth Models
- Sparse Flows: Pruning Continuous-depth Models
- Continuous-Depth Neural Models for Dynamic Graph Prediction
- Differentiable Multiple Shooting Layers
- Freudian and Newtonian Recurrent Cell for Sequential Recommendation