2 citations · 3 across the 7 of their papers we have counts for
1 paper · 1 filter
Xu Owen He
The feedforward (FFW) layers in standard transformer architectures incur a linear increase in computational costs and activation memory as the hidden layer width grows. Sparse mixt…