Extending Sparse Tensor Accelerators to Support Multiple Compression Formats
arXiv:2103.10452
Abstract
Sparsity, which occurs in both scientific applications and Deep Learning (DL) models, has been a key target of optimization within recent ASIC accelerators due to the potential memory and compute savings. These applications use data stored in a variety of compression formats. We demonstrate that both the compactness of different compression formats and the compute efficiency of the algorithms enabled by them vary across tensor dimensions and amount of sparsity. Since DL and scientific workloads span across all sparsity regions, there can be numerous format combinations for optimizing memory and compute efficiency. Unfortunately, many proposed accelerators operate on one or two fixed format combinations. This work proposes hardware extensions to accelerators for supporting numerous format combinations seamlessly and demonstrates ~4X speedup over performing format conversions in software.
arXiv commentAccepted for publication at the 35th IEEE International Parallel & Distributed Processing Symposium (IPDPS 2021)
References in corpus (5)
- To prune, or not to prune: exploring the efficacy of pruning for model compression
- SpArch: Efficient Architecture for Sparse Matrix Multiplication
- Format Abstraction for Sparse Tensor Algebra Compilers
- SMASH: Co-designing Software Compression and Hardware-Accelerated Indexing for Efficient Sparse Matrix Operations
- A Unified Optimization Approach for Sparse Tensor Operations on GPUs