3 papers
cs.LG2026
CommFuse: Hiding Tail Latency via Communication Decomposition and Fusion for Distributed LLM Training
Rezaul Karim, Austin Wen, Wang Zongzuo +3
The rapid growth in the size of large language models has necessitated the partitioning of computational workloads across accelerators such as GPUs, TPUs, and NPUs. However, these…
cs.LG2024
Accelerating the Low-Rank Decomposed Models
Habib Hajimolahoseini, Walid Ahmed, Austin Wen +1
Tensor decomposition is a mathematically supported technique for data compression. It consists of applying some kind of a Low Rank Decomposition technique on the tensors or matrice…
cs.CV2024
Is 3D Convolution with 5D Tensors Really Necessary for Video Analysis?
Habib Hajimolahoseini, Walid Ahmed, Austin Wen +1
In this paper, we present a comprehensive study and propose several novel techniques for implementing 3D convolutional blocks using 2D and/or 1D convolutions with only 4D and/or 3D…