1 citations · 3 across the 7 of their papers we have counts for
4 papers · 1 filter
Change of Thought: Adaptive Test-Time Computation
Mrinal Mathur, Mike Doan, Barak Pearlmutter +1
Transformers evaluated in a single, fixed-depth pass are provably limited in expressive power to the constant-depth circuit class TC0. Running a Transformer autoregressively remove…
Recent Trends in Modelling the Continuous Time Series using Deep Learning: A Survey
Mansura Habiba, Barak A. Pearlmutter, Mehrdad Maleki
Continuous-time series is essential for different modern application areas, e.g. healthcare, automobile, energy, finance, Internet of things (IoT) and other related areas. Differen…
Second-Order Forward-Mode Automatic Differentiation for Optimization
Adam D. Cobb, Atılım Güneş Baydin, Barak A. Pearlmutter +1
This paper introduces a second-order hyperplane search, a novel optimization step that generalizes a second-order line search from a line to a -dimensional hyperplane. This, com…
Low-Rank Learning by Design: the Role of Network Architecture and Activation Linearity in Gradient Rank Collapse
Bradley T. Baker, Barak A. Pearlmutter, Robyn Miller +2
Our understanding of learning dynamics of deep neural networks (DNNs) remains incomplete. Recent research has begun to uncover the mathematical principles underlying these networks…