10 citations · 13 across the 5 of their papers we have counts for
6 papers · 1 filter
Pushback in a Haystack: Language Models Respond to Factual Errors Ambiguously and Ineffectively
Li Lucy, Myra Cheng, Svenja Guhr +7
The sycophancy of language models, or their failure to push back against faulty user input, is now widely studied. However, if LMs do push back, is sycophancy solved? We show that…
The Aftermath of DrawEduMath: Vision Language Models Underperform with Struggling Students and Misdiagnose Errors
Li Lucy, Albert Zhang, Nathan Anderson +2
Effective mathematics education requires identifying and responding to students' mistakes. For AI to support pedagogical applications, models must perform well across different lev…
DrawEduMath: Evaluating Vision Language Models with Expert-Annotated Students' Hand-Drawn Math Images
Sami Baral, Li Lucy, Ryan Knight +4
In real-world settings, vision language models (VLMs) should robustly handle naturalistic, noisy visual content as well as domain-specific language and concepts. For example, K-12…
Mathfish: Evaluating Language Model Math Reasoning via Grounding in Educational Curricula
Li Lucy, Tal August, Rose E. Wang +3
To ensure that math curriculum is grade-appropriate and aligns with critical skills or concepts in accordance with educational standards, pedagogical experts can spend months caref…
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
Luca Soldaini, Rodney Kinney, Akshita Bhagia +33
Information about pretraining corpora used to train the current best-performing language models is seldom discussed: commercial models rarely detail their data, and even open model…
"One-Size-Fits-All"? Examining Expectations around What Constitute "Fair" or "Good" NLG System Behaviors
Li Lucy, Su Lin Blodgett, Milad Shokouhi +2
Fairness-related assumptions about what constitute appropriate NLG system behaviors range from invariance, where systems are expected to behave identically for social groups, to ad…