Showing 2023Show all
3 papers · 1 filter
cs.LG2023
GQKVA: Efficient Pre-training of Transformers by Grouping Queries, Keys, and Values
Farnoosh Javadi, Walid Ahmed, Habib Hajimolahoseini +7
Massive transformer-based models face several challenges, including slow and computationally intensive pre-training and over-parametrization. This paper addresses these challenges…
cs.LG2023
SwiftLearn: A Data-Efficient Training Method of Deep Learning Models using Importance Sampling
Habib Hajimolahoseini, Omar Mohamed Awad, Walid Ahmed +8
In this paper, we present SwiftLearn, a data-efficient approach to accelerate training of deep learning models using a subset of data samples selected during the warm-up stages of…
cs.CV2023
Speeding up Resnet Architecture with Layers Targeted Low Rank Decomposition
Walid Ahmed, Habib Hajimolahoseini, Austin Wen +1
Compression of a neural network can help in speeding up both the training and the inference of the network. In this research, we study applying compression using low rank decomposi…