Syntax-guided Localized Self-attention by Constituency Syntactic Distance

Shengyuan Hou, Jushi Kai, Haotian Xue +5

Recent works have revealed that Transformers are implicitly learning the syntactic information in its lower layers from data, albeit is highly dependent on the quality and scale of…