activity
20222026
most citedBoosting Distributed Training Performance of the Unpadded BERT Model

4 citations · 6 across the 6 of their papers we have counts for

collaborators
Showing 2024Show all

1 paper · 1 filter

cs.LG2024

FlashMask: Efficient and Rich Mask Extension of FlashAttention

Guoxia Wang, Jinle Zeng, Xiyuan Xiao +7

The computational and memory demands of vanilla attention scale quadratically with the sequence length , posing significant challenges for processing long sequences in Transform…