5 citations · 5 across the 1 of their papers we have counts for
3 papers
Refining Targeted Syntactic Evaluation of Language Models
Benjamin Newman, Kai-Siang Ang, Julia Gong +1
Targeted syntactic evaluation of subject-verb number agreement in English (TSE) evaluates language models' syntactic knowledge using hand-crafted minimal pairs of sentences that di…
The EOS Decision and Length Extrapolation
Benjamin Newman, John Hewitt, Percy Liang +1
Extrapolation to unseen sequence lengths is a challenge for neural generative models of language. In this work, we characterize the effect on length extrapolation of a modeling dec…
Communication-based Evaluation for Natural Language Generation
Benjamin Newman, Reuben Cohn-Gordon, Christopher Potts
Natural language generation (NLG) systems are commonly evaluated using n-gram overlap measures (e.g. BLEU, ROUGE). These measures do not directly capture semantics or speaker inten…