3 papers
cs.LG2024
Approximate Top- for Increased Parallelism
Oscar Key, Luka Ribar, Alberto Cattaneo +2
We present an evaluation of bucketed approximate top- algorithms. Computing top- exactly suffers from limited parallelism, because the largest values must be aggregated a…
cs.LG2023
PopSparse: Accelerated block sparse matrix multiplication on IPU
Zhiyi Li, Douglas Orr, Valeriu Ohan +5
Reducing the computational cost of running large scale neural networks using sparsity has attracted great attention in the deep learning community. While much success has been achi…
cs.LG2023
Unit Scaling: Out-of-the-Box Low-Precision Training
Charlie Blake, Douglas Orr, Carlo Luschi
We present unit scaling, a paradigm for designing deep learning models that simplifies the use of low-precision number formats. Training in FP16 or the recently proposed FP8 format…