2 papers
cs.CL2024
Natural Language Processing RELIES on Linguistics
Juri Opitz, Shira Wein, Nathan Schneider
Large Language Models (LLMs) have become capable of generating highly fluent text in certain languages, without modules specially designed to capture grammar or semantic coherence.…
cs.CL2023
Syntactic Inductive Bias in Transformer Language Models: Especially Helpful for Low-Resource Languages?
Luke Gessler, Nathan Schneider
A line of work on Transformer-based language models such as BERT has attempted to use syntactic inductive bias to enhance the pretraining process, on the theory that building synta…