31 citations · 45 across the 5 of their papers we have counts for
Showing 2020Show all
2 papers · 1 filter
cs.LG2020★ 7 cited
Repulsive Attention: Rethinking Multi-head Attention as Bayesian Inference
Bang An, Jie Lyu, Zhenyi Wang +6
The neural attention mechanism plays an important role in many natural language processing applications. In particular, the use of multi-head attention extends single-head attentio…
cs.CL2020★ 7 cited
Calibrated Language Model Fine-Tuning for In- and Out-of-Distribution Data
Lingkai Kong, Haoming Jiang, Yuchen Zhuang +3
Fine-tuned pre-trained language models can suffer from severe miscalibration for both in-distribution and out-of-distribution (OOD) data due to over-parameterization. To mitigate t…