1 citations · 2 across the 2 of their papers we have counts for
2 papers
cs.LG2023★ 1 cited
Finding the Pillars of Strength for Multi-Head Attention
Jinjie Ni, Rui Mao, Zonglin Yang +2
Recent studies have revealed some issues of Multi-Head Attention (MHA), e.g., redundancy and over-parameterization. Specifically, the heads of MHA were originally designed to atten…
cs.CL2023★ 1 cited
Adaptive Knowledge Distillation between Text and Speech Pre-trained Models
Jinjie Ni, Yukun Ma, Wen Wang +7
Learning on a massive amount of speech corpus leads to the recent success of many self-supervised speech models. With knowledge distillation, these models may also benefit from the…