805 citations · 1.2k across the 10 of their papers we have counts for
10 papers
A Scalable NorthPole System with End-to-End Vertical Integration for Low-Latency and Energy-Efficient LLM Inference
Michael V. DeBole, Rathinakumar Appuswamy, Neil McGlohon +30
A vertically integrated, end-to-end, research prototype system combines 288 NorthPole neural inference accelerator cards, offline training algorithms, a high-performance runtime st…
SiLQ: Simple Large Language Model Quantization-Aware Training
Steven K. Esser, Jeffrey L. McKinstry, Deepika Bablani +2
Large language models can be quantized to reduce inference time latency, model size, and energy consumption, thereby delivering a better user experience at lower cost. A challenge…
Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling
Xiuying Wei, Yunchen Zhang, Yuhang Li +4
Post-training quantization~(PTQ) of transformer language models faces significant challenges due to the existence of detrimental outliers in activations. We observe that these outl…
Efficient and Effective Methods for Mixed Precision Neural Network Quantization for Faster, Energy-efficient Inference
Deepika Bablani, Jeffrey L. Mckinstry, Steven K. Esser +2
For efficient neural network inference, it is desirable to achieve state-of-the-art accuracy with the simplest networks requiring the least computation, memory, and power. Quantizi…
Learned Step Size Quantization
Steven K. Esser, Jeffrey L. McKinstry, Deepika Bablani +2
Deep networks run with low precision operations at inference time offer power and space advantages over high precision alternatives, but need to overcome the challenge of maintaini…
Low Precision Policy Distillation with Application to Low-Power, Real-time Sensation-Cognition-Action Loop with Neuromorphic Computing
Jeffrey L Mckinstry, Davis R. Barch, Deepika Bablani +5
Low precision networks in the reinforcement learning (RL) setting are relatively unexplored because of the limitations of binary activations for function approximation. Here, in th…
Discovering Low-Precision Networks Close to Full-Precision Networks for Efficient Embedded Inference
Jeffrey L. McKinstry, Steven K. Esser, Rathinakumar Appuswamy +4
To realize the promise of ubiquitous embedded deep network inference, it is essential to seek limits of energy and area efficiency. To this end, low-precision networks offer tremen…
Structured Convolution Matrices for Energy-efficient Deep learning
Rathinakumar Appuswamy, Tapan Nayak, John Arthur +6
We derive a relationship between network representation in energy-efficient neuromorphic architectures and block Toplitz convolutional matrices. Inspired by this connection, we dev…
Deep neural networks are robust to weight binarization and other non-linear distortions
Paul Merolla, Rathinakumar Appuswamy, John Arthur +2
Recent results show that deep neural networks achieve excellent performance even when, during training, weights are quantized and projected to a binary representation. Here, we sho…
Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing
Steven K. Esser, Paul A. Merolla, John V. Arthur +13
Deep networks are now able to achieve human-level performance on a broad spectrum of recognition tasks. Independently, neuromorphic computing has now demonstrated unprecedented ene…