Revisiting Over-smoothing in BERT from the Perspective of Graph

Han Shi, Jiahui Gao, Hang Xu +5

Recently over-smoothing phenomenon of Transformer-based models is observed in both vision and language fields. However, no existing work has delved deeper to further investigate th…