3 citations · 10 across the 9 of their papers we have counts for
3 papers · 1 filter
SparQ Attention: Bandwidth-Efficient LLM Inference
Luka Ribar, Ivan Chelombiev, Luke Hudlass-Galley +3
The computational difficulties of large language model (LLM) inference remain a significant obstacle to their widespread deployment. The need for many applications to support long…
PopSparse: Accelerated block sparse matrix multiplication on IPU
Zhiyi Li, Douglas Orr, Valeriu Ohan +5
Reducing the computational cost of running large scale neural networks using sparsity has attracted great attention in the deep learning community. While much success has been achi…
Unit Scaling: Out-of-the-Box Low-Precision Training
Charlie Blake, Douglas Orr, Carlo Luschi
We present unit scaling, a paradigm for designing deep learning models that simplifies the use of low-precision number formats. Training in FP16 or the recently proposed FP8 format…