4 papers
Is 3D Convolution with 5D Tensors Really Necessary for Video Analysis?
Habib Hajimolahoseini, Walid Ahmed, Austin Wen +1
In this paper, we present a comprehensive study and propose several novel techniques for implementing 3D convolutional blocks using 2D and/or 1D convolutions with only 4D and/or 3D…
Single Parent Family: A Spectrum of Family Members from a Single Pre-Trained Foundation Model
Habib Hajimolahoseini, Mohammad Hassanpour, Foozhan Ataiefard +2
This paper introduces a novel method of Progressive Low Rank Decomposition (PLRD) tailored for the compression of large language models. Our approach leverages a pre-trained model,…
SkipViT: Speeding Up Vision Transformers with a Token-Level Skip Connection
Foozhan Ataiefard, Walid Ahmed, Habib Hajimolahoseini +7
Vision transformers are known to be more computationally and data-intensive than CNN models. These transformer models such as ViT, require all the input image tokens to learn the r…
Speeding up Resnet Architecture with Layers Targeted Low Rank Decomposition
Walid Ahmed, Habib Hajimolahoseini, Austin Wen +1
Compression of a neural network can help in speeding up both the training and the inference of the network. In this research, we study applying compression using low rank decomposi…