A Systematic Study of Compositional Syntactic Transformer Language Models

Yida Zhao, Hao Xve, Xiang Hu +1

Syntactic language models (SLMs) enhance Transformers by incorporating syntactic biases through the modeling of linearized syntactic parse trees alongside surface sentences. This p…