Transfer Learning using Neural Ordinary Differential Equations
arXiv:2001.07342
Abstract
A concept of using Neural Ordinary Differential Equations(NODE) for Transfer Learning has been introduced. In this paper we use the EfficientNets to explore transfer learning on CIFAR-10 dataset. We use NODE for fine-tuning our model. Using NODE for fine tuning provides more stability during training and validation.These continuous depth blocks can also have a trade off between numerical precision and speed .Using Neural ODEs for transfer learning has resulted in much stable convergence of the loss function.
References in corpus (7)
- Adam: A Method for Stochastic Optimization
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
- Augmented Neural ODEs
- Probabilistic ODE Solvers with Runge-Kutta Means
- ANODEV2: A Coupled Neural ODE Evolution Framework
- Fast derivatives of likelihood functionals for ODE based models using adjoint-state method