Understanding Syntactic Generalization in Structure-inducing Language Models

David Arps, Hassan Sajjad, Laura Kallmeyer

Structure-inducing Language Models (SiLM) are trained on a self-supervised language modeling task, and induce a hierarchical sentence representation as a byproduct when processing…