24 citations · 64 across the 15 of their papers we have counts for
10 papers · 1 filter
Low-Rank Learning by Design: the Role of Network Architecture and Activation Linearity in Gradient Rank Collapse
Bradley T. Baker, Barak A. Pearlmutter, Robyn Miller +2
Our understanding of learning dynamics of deep neural networks (DNNs) remains incomplete. Recent research has begun to uncover the mathematical principles underlying these networks…
Gradients without Backpropagation
Atılım Güneş Baydin, Barak A. Pearlmutter, Don Syme +2
Using backpropagation to compute gradients of objective functions for optimization has remained a mainstay of machine learning. Backpropagation, or reverse-mode differentiation, is…
Continuous Convolutional Neural Networks: Coupled Neural PDE and ODE
Mansura Habiba, Barak A. Pearlmutter
Recent work in deep learning focuses on solving physical systems in the Ordinary Differential Equation or Partial Differential Equation. This current work proposed a variant of Con…
Neural Network based on Automatic Differentiation Transformation of Numeric Iterate-to-Fixedpoint
Mansura Habiba, Barak A. Pearlmutter
This work proposes a Neural Network model that can control its depth using an iterate-to-fixed-point operator. The architecture starts with a standard layered Network but with adde…
HeunNet: Extending ResNet using Heun's Methods
Mehrdad Maleki, Mansura Habiba, Barak A. Pearlmutter
There is an analogy between the ResNet (Residual Network) architecture for deep neural networks and an Euler solver for an ODE. The transformation performed by each layer resembles…
Neural ODEs for Informative Missingness in Multivariate Time Series
Mansura Habiba, Barak A. Pearlmutter
Informative missingness is unavoidable in the digital processing of continuous time series, where the value for one or more observations at different time points are missing. Such…