15 citations · 15 across the 2 of their papers we have counts for
2 papers
cs.CL2023★ 15 cited
DropDim: A Regularization Method for Transformer Networks
Hao Zhang, Dan Qu, Keji Shao +1
We introduceDropDim, a structured dropout method designed for regularizing the self-attention mechanism, which is a key component of the transformer. In contrast to the general dro…
cs.CL2023
Decouple Non-parametric Knowledge Distillation For End-to-end Speech Translation
Hao Zhang, Nianwen Si, Yaqi Chen +4
Existing techniques often attempt to make knowledge transfer from a powerful machine translation (MT) to speech translation (ST) model with some elaborate techniques, which often r…