212 citations · 479 across the 14 of their papers we have counts for
6 papers · 1 filter
Debugging using Orthogonal Gradient Descent
Narsimha Chilkuri, Chris Eliasmith
In this report we consider the following problem: Given a trained model that is partially faulty, can we correct its behaviour without having to train the model from scratch? In ot…
Language Modeling using LMUs: 10x Better Data Efficiency or Improved Scaling Compared to Transformers
Narsimha Chilkuri, Eric Hunsberger, Aaron Voelker +2
Recent studies have demonstrated that the performance of transformers on the task of language modeling obeys a power-law relationship with model size over six orders of magnitude.…
Parallelizing Legendre Memory Unit Training
Narsimha Chilkuri, Chris Eliasmith
Recently, a new recurrent neural network (RNN) named the Legendre Memory Unit (LMU) was proposed and shown to achieve state-of-the-art performance on several benchmark datasets. He…
A Spike in Performance: Training Hybrid-Spiking Neural Networks with Quantized Activation Functions
Aaron R. Voelker, Daniel Rasmussen, Chris Eliasmith
The machine learning community has become increasingly interested in the energy efficiency of neural networks. The Spiking Neural Network (SNN) is a promising approach to energy-ef…
Benchmarking Keyword Spotting Efficiency on Neuromorphic Hardware
Peter Blouw, Xuan Choo, Eric Hunsberger +1
Using Intel's Loihi neuromorphic research chip and ABR's Nengo Deep Learning toolkit, we analyze the inference speed, dynamic power consumption, and energy cost per inference of a…
Spiking Deep Networks with LIF Neurons
Eric Hunsberger, Chris Eliasmith
We train spiking deep networks using leaky integrate-and-fire (LIF) neurons, and achieve state-of-the-art results for spiking networks on the CIFAR-10 and MNIST datasets. This demo…