33 citations · 91 across the 17 of their papers we have counts for
17 papers
DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers
Sayeh Sharify, Mahsa Salmani, Hesham Mostafa
Diffusion Transformers (DiTs) achieve state-of-the-art image generation quality but incur substantial memory and computational costs at inference. While aggressive Post-Training Qu…
MF-QAT: Multi-Format Quantization-Aware Training for Elastic Inference
Zifei Xu, Sayeh Sharify, Hesham Mostafa
Quantization-aware training (QAT) is typically performed for a single target numeric format, while practical deployments often need to choose numerical precision at inference time…
Early Attentive Sparsification Accelerates Neural Speech Transcription
Zifei Xu, Sayeh Sharify, Hesham Mostafa +3
Transformer-based neural speech processing has achieved state-of-the-art performance. Since speech audio signals are known to be highly compressible, here we seek to accelerate neu…
ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residuals
Utkarsh Saxena, Sayeh Sharify, Kaushik Roy +1
Post-training quantization (PTQ) of large language models (LLMs) holds the promise in reducing the prohibitive computational cost at inference time. Quantization of all weight, act…
Understanding the Difficulty of Low-Precision Post-Training Quantization for LLMs
Zifei Xu, Sayeh Sharify, Wanzin Yazar +2
Large language models of high parameter counts are computationally expensive, yet can be made much more efficient by compressing their weights to very low numerical precision. This…
Scaling Laws for Post Training Quantized Large Language Models
Zifei Xu, Alexander Lan, Wanzin Yazar +3
Generalization abilities of well-trained large language models (LLMs) are known to scale predictably as a function of model size. In contrast to the existence of practical scaling…
Post Training Quantization of Large Language Models with Microscaling Formats
Sayeh Sharify, Utkarsh Saxena, Zifei Xu +3
Large Language Models (LLMs) have distinguished themselves with outstanding performance in complex language modeling tasks, yet they come with significant computational and storage…
Self-Selected Attention Span for Accelerating Large Language Model Inference
Tian Jin, Wanzin Yazar, Zifei Xu +2
Large language models (LLMs) can solve challenging tasks. However, their inference computation on modern GPUs is highly inefficient due to the increasing number of tokens they must…
Mixed-Precision Quantization for Deep Vision Models with Integer Quadratic Programming
Zihao Deng, Sayeh Sharify, Xin Wang +1
Quantization is a widely used technique to compress neural networks. Assigning uniform bit-widths across all layers can result in significant accuracy degradation at low precision…
Laconic Deep Learning Computing
Sayeh Sharify, Mostafa Mahmoud, Alberto Delmas Lascorz +2
We motivate a method for transparently identifying ineffectual computations in unmodified Deep Learning models and without affecting accuracy. Specifically, we show that if we deco…
DPRed: Making Typical Activation and Weight Values Matter In Deep Learning Computing
Alberto Delmas, Sayeh Sharify, Patrick Judd +3
We show that selecting a single data type (precision) for all values in Deep Neural Networks, even if that data type is different per layer, amounts to worst case design. Much shor…
Bit-Tactical: Exploiting Ineffectual Computations in Convolutional Neural Networks: Which, Why, and How
Alberto Delmas, Patrick Judd, Dylan Malone Stuart +5
We show that, during inference with Convolutional Neural Networks (CNNs), more than 2x to $8x ineffectual work can be exposed if instead of targeting those weights and activations…
Tartan: Accelerating Fully-Connected and Convolutional Layers in Deep Learning Networks by Exploiting Numerical Precision Variability
Alberto Delmas, Sayeh Sharify, Patrick Judd +1
Tartan (TRT), a hardware accelerator for inference with Deep Neural Networks (DNNs), is presented and evaluated on Convolutional Neural Networks. TRT exploits the variable per laye…
Loom: Exploiting Weight and Activation Precisions to Accelerate Convolutional Neural Networks
Sayeh Sharify, Alberto Delmas Lascorz, Kevin Siu +2
Loom (LM), a hardware inference accelerator for Convolutional Neural Networks (CNNs) is presented. In LM every bit of data precision that can be saved translates to proportional pe…
Dynamic Stripes: Exploiting the Dynamic Precision Requirements of Activation Values in Neural Networks
Alberto Delmas, Patrick Judd, Sayeh Sharify +1
Stripes is a Deep Neural Network (DNN) accelerator that uses bit-serial computation to offer performance that is proportional to the fixed-point precision of the activation values.…
Cnvlutin2: Ineffectual-Activation-and-Weight-Free Deep Neural Network Computing
Patrick Judd, Alberto Delmas, Sayeh Sharify +1
We discuss several modifications and extensions over the previous proposed Cnvlutin (CNV) accelerator for convolutional and fully-connected layers of Deep Learning Network. We firs…
Bit-pragmatic Deep Neural Network Computing
J. Albericio, P. Judd, A. Delmás +2
We quantify a source of ineffectual computations when processing the multiplications of the convolutional layers in Deep Neural Networks (DNNs) and propose Pragmatic (PRA), an arch…