DiffEqFlux.jl - A Julia Library for Neural Differential Equations
arXiv:1902.02376
Abstract
DiffEqFlux.jl is a library for fusing neural networks and differential equations. In this work we describe differential equations from the viewpoint of data science and discuss the complementary nature between machine learning models and differential equations. We demonstrate the ability to incorporate DifferentialEquations.jl-defined differential equation problems into a Flux-defined neural network, and vice versa. The advantages of being able to use the entire DifferentialEquations.jl suite for this purpose is demonstrated by counter examples where simple integration strategies fail, but the sophisticated integration strategies provided by the DifferentialEquations.jl library succeed. This is followed by a demonstration of delay differential equations and stochastic differential equations inside of neural networks. We show high-level functionality for defining neural ordinary differential equations (neural networks embedded into the differential equation) and describe the extra models in the Flux model zoo which includes neural stochastic differential equations. We conclude by discussing the various adjoint methods used for backpropogation of the differential equation solvers. DiffEqFlux.jl is an important contribution to the area, as it allows the full weight of the differential equation solvers developed from decades of research in the scientific computing field to be readily applied to the challenges posed by machine learning and data science.
arXiv commentJulialang Blog post, DiffEqFlux.jl
References in corpus (3)
Cited by in corpus (35)
- Solving nonlinear differential equations with differentiable quantum circuits
- Stiff Neural Ordinary Differential Equations
- Neural Jump Stochastic Differential Equations
- Julia Language in Machine Learning: Algorithms, Applications, and Open Issues
- Learning Differential Equations that are Easy to Solve
- Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit
- Neural Network aided quarantine control model estimation of global Covid-19 spread
- Neural Network aided quarantine control model estimation of COVID spread in Wuhan, China
- Using neural networks to accelerate the solution of the Boltzmann equation
- Reducing the complexity of chemical networks via interpretable autoencoders
- Multiple shooting for training neural differential equations on time series
- Robust quantum gates using smooth pulses and physics-informed neural networks
- Discretize-Optimize vs. Optimize-Discretize for Time-Series Regression and Continuous Normalizing Flows
- Earth System Data Cubes: Avenues for advancing Earth system research
- PowerDynamics.jl -- An experimentally validated open-source package for the dynamical analysis of power grids
- Approximation Capabilities of Neural ODEs and Invertible Residual Networks
- Data-Driven Surrogates of Rotating Detonation Engine Physics with Neural ODEs and High-Speed Camera Footage
- NetworkDynamics.jl -- Composing and simulating complex networks in Julia
- TorchDyn: A Neural Differential Equations Library
- Inference of neutrino flavor evolution through data assimilation and neural differential equations
- Two-qubit CZ gates robust against charge noise in silicon while compensating for crosstalk using neural network
- NeuralFMU: Towards Structural Integration of FMUs into Neural Networks
- Learning Stable Galerkin Models of Turbulence with Differentiable Programming
- Variational optimization and data assimilation in chaotic time-delayed systems with automatic-differentiated shadowing sensitivity
- Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristics
- System Identification for Hybrid Systems using Neural Networks
- Physics-Informed Machine Learning Simulator for Wildfire Propagation
- A research framework for writing differentiable PDE discretizations in JAX
- Deep learning and differential equations for modeling changes in individual-level latent dynamics between observation periods
- "Hey, that's not an ODE": Faster ODE Adjoints via Seminorms
- Augmenting Neural Differential Equations to Model Unknown Dynamical Systems with Incomplete State Information
- ChronoMID - Cross-Modal Neural Networks for 3-D Temporal Medical Imaging Data
- An Open Source Software Stack for Tuning the Dynamical Behavior of Complex Power Systems
- Differentiable Multiple Shooting Layers
- Pathogen.jl: Infectious Disease Transmission Network Modelling with Julia