80 citations · 93 across the 7 of their papers we have counts for
8 papers
ECHO-LLaMA: Efficient Caching for High-Performance LLaMA Training
Maryam Dialameh, Rezaul Karim, Hossein Rajabzadeh +5
This paper introduces ECHO-LLaMA, an efficient LLaMA architecture designed to improve both the training speed and inference throughput of LLaMA architectures while maintaining its…
SkipViT: Speeding Up Vision Transformers with a Token-Level Skip Connection
Foozhan Ataiefard, Walid Ahmed, Habib Hajimolahoseini +7
Vision transformers are known to be more computationally and data-intensive than CNN models. These transformer models such as ViT, require all the input image tokens to learn the r…
GQKVA: Efficient Pre-training of Transformers by Grouping Queries, Keys, and Values
Farnoosh Javadi, Walid Ahmed, Habib Hajimolahoseini +7
Massive transformer-based models face several challenges, including slow and computationally intensive pre-training and over-parametrization. This paper addresses these challenges…
SwiftLearn: A Data-Efficient Training Method of Deep Learning Models using Importance Sampling
Habib Hajimolahoseini, Omar Mohamed Awad, Walid Ahmed +8
In this paper, we present SwiftLearn, a data-efficient approach to accelerate training of deep learning models using a subset of data samples selected during the warm-up stages of…
Improving Resnet-9 Generalization Trained on Small Datasets
Omar Mohamed Awad, Habib Hajimolahoseini, Michael Lim +4
This paper presents our proposed approach that won the first prize at the ICLR competition on Hardware Aware Efficient Training. The challenge is to achieve the highest possible ac…
FPRaker: A Processing Element For Accelerating Neural Network Training
Omar Mohamed Awad, Mostafa Mahmoud, Isak Edo +5
We present FPRaker, a processing element for composing training accelerators. FPRaker processes several floating-point multiply-accumulation operations concurrently and accumulates…
TensorDash: Exploiting Sparsity to Accelerate Deep Neural Network Training and Inference
Mostafa Mahmoud, Isak Edo, Ali Hadi Zadeh +4
TensorDash is a hardware level technique for enabling data-parallel MAC units to take advantage of sparsity in their input operand streams. When used to compose a hardware accelera…
GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inference
Ali Hadi Zadeh, Isak Edo, Omar Mohamed Awad +1
Attention-based models have demonstrated remarkable success in various natural language understanding tasks. However, efficient execution remains a challenge for these models which…