221 citations · 395 across the 21 of their papers we have counts for
11 papers · 1 filter
JaxPruner: A concise library for sparsity research
Joo Hyung Lee, Wonpyo Park, Nicole Mitchell +22
This paper introduces JaxPruner, an open-source JAX-based pruning and sparse training library for machine learning research. JaxPruner aims to accelerate research on sparse neural…
STEP: Learning N:M Structured Sparsity Masks from Scratch with Precondition
Yucheng Lu, Shivani Agrawal, Suvinay Subramanian +3
Recent innovations on hardware (e.g. Nvidia A100) have motivated learning N:M structured sparsity masks from scratch for fast model inference. However, state-of-the-art learning re…
GRANITE: A Graph Neural Network Model for Basic Block Throughput Estimation
Ondrej Sykora, Phitchaya Mangpo Phothilimthana, Charith Mendis +1
Analytical hardware performance models yield swift estimation of desired hardware performance metrics. However, developing these analytical models for modern processors with sophis…
Training Recipe for N:M Structured Sparsity with Decaying Pruning Mask
Sheng-Chun Kao, Amir Yazdanbakhsh, Suvinay Subramanian +3
Sparsity has become one of the promising methods to compress and accelerate Deep Neural Networks (DNNs). Among different categories of sparsity, structured sparsity has gained more…
Sparse Attention Acceleration with Synergistic In-Memory Pruning and On-Chip Recomputation
Amir Yazdanbakhsh, Ashkan Moradifirouzabadi, Zheng Li +1
As its core computation, a self-attention mechanism gauges pairwise correlations across the entire input sequence. Despite favorable performance, calculating pairwise correlations…
FLAT: An Optimized Dataflow for Mitigating Attention Bottlenecks
Sheng-Chun Kao, Suvinay Subramanian, Gaurav Agrawal +2
Attention mechanisms, primarily designed to capture pairwise correlations between words, have become the backbone of machine learning, expanding beyond natural language processing…
Rethinking Co-design of Neural Architectures and Hardware Accelerators
Yanqi Zhou, Xuanyi Dong, Berkin Akin +7
Neural architectures and hardware accelerators have been two driving forces for the progress in deep learning. Previous works typically attempt to optimize hardware given a fixed m…
Apollo: Transferable Architecture Exploration
Amir Yazdanbakhsh, Christof Angermueller, Berkin Akin +7
The looming end of Moore's Law and ascending use of deep learning drives the design of custom accelerators that are optimized for specific neural architectures. Architecture explor…
An Evaluation of Edge TPU Accelerators for Convolutional Neural Networks
Kiran Seshadri, Berkin Akin, James Laudon +2
Edge TPUs are a domain of accelerators for low-power, edge devices and are widely used in various Google products such as Coral and Pixel devices. In this paper, we first discuss t…
Chameleon: Adaptive Code Optimization for Expedited Deep Neural Network Compilation
Byung Hoon Ahn, Prannoy Pilligundla, Amir Yazdanbakhsh +1
Achieving faster execution with shorter compilation time can foster further diversity and innovation in neural networks. However, the current paradigm of executing neural networks…
ReLeQ: A Reinforcement Learning Approach for Deep Quantization of Neural Networks
Ahmed T. Elthakeb, Prannoy Pilligundla, FatemehSadat Mireshghallah +2
Deep Neural Networks (DNNs) typically require massive amount of computation resource in inference tasks for computer vision applications. Quantization can significantly reduce DNN…