121 citations · 181 across the 19 of their papers we have counts for
16 papers · 1 filter
Pigeonholing: how bad prompts hurt models, causing collapse and mistakes
Hyunji Nam, Keertana Chidambaram, Dorottya Demszky +1
While in-context learning is generally shown to be effective in Large Language Models (LLMs), bad contexts can cause performance degradation and mode collapse, a phenomenon we call…
From Scoring to Explanations: Evaluating SHAP and LLM Rationales for Rubric-based Teaching Quality Assessment
Ivo Bueno, Babette Bühler, Philipp Stark +5
Automated scoring models are increasingly used to assign rubric-based quality ratings to complex language performances, including classroom transcripts, yet they typically provide…
Marked Pedagogies: Examining Linguistic Biases in Personalized Automated Writing Feedback
Mei Tan, Lena Phalen, Dorottya Demszky
Effective personalized feedback is critical to students' literacy development. Though LLM-powered tools now promise to automate such feedback at scale, LLMs are not language-neutra…
IDEAlign: Comparing Ideas of Large Language Models to Domain Expert
Hyunji Nam, Lucia Langlois, James Malamut +2
Large language models (LLMs) are increasingly used to produce open-ended, interpretive annotations, yet there is no validated, scalable measure of idea-level similarity to expert a…
EduCoder: An Open-Source Annotation System for Education Transcript Data
Saad Ashraf, James Malamut, Vishal Kumar +7
We introduce EduCoder, a domain-specialized tool designed to support utterance-level annotation of educational dialogue. While general-purpose text annotation tools for NLP and qua…
Continued Pretraining for Domain Adaptation of Wav2vec2.0 in Automatic Speech Recognition for Elementary Math Classroom Settings
Ahmed Adel Attia, Dorottya Demszky, Tolulope Ogunremi +2
Creating Automatic Speech Recognition (ASR) systems that are robust and resilient to classroom conditions is paramount to the development of AI tools to aid teachers and students.…
SIGHT: A Large Annotated Dataset on Student Insights Gathered from Higher Education Transcripts
Rose E. Wang, Pawan Wirawarn, Noah Goodman +1
Lectures are a learning experience for both students and teachers. Students learn from teachers about the subject material, while teachers learn from students about how to refine t…
Is ChatGPT a Good Teacher Coach? Measuring Zero-Shot Performance For Scoring and Providing Actionable Insights on Classroom Instruction
Rose E. Wang, Dorottya Demszky
Coaching, which involves classroom observation and expert feedback, is a widespread and fundamental part of teacher training. However, the majority of teachers do not have access t…
MD3: The Multi-Dialect Dataset of Dialogues
Jacob Eisenstein, Vinodkumar Prabhakaran, Clara Rivera +2
We introduce a new dataset of conversational speech representing English from India, Nigeria, and the United States. The Multi-Dialect Dataset of Dialogues (MD3) strikes a new bala…
The NCTE Transcripts: A Dataset of Elementary Math Classroom Transcripts
Dorottya Demszky, Heather Hill
Classroom discourse is a core medium of instruction - analyzing it can provide a window into teaching and learning as well as driving the development of new tools for improving ins…
Measuring Conversational Uptake: A Case Study on Student-Teacher Interactions
Dorottya Demszky, Jing Liu, Zid Mancenido +4
In conversation, uptake happens when a speaker builds on the contribution of their interlocutor by, for example, acknowledging, repeating or reformulating what they have said. In e…
Learning to Recognize Dialect Features
Dorottya Demszky, Devyani Sharma, Jonathan H. Clark +2
Building NLP systems that serve everyone requires accounting for dialect differences. But dialects are not monolithic entities: rather, distinctions between and within dialects are…
The Role of Verb Semantics in Hungarian Verb-Object Order
Dorottya Demszky, László Kálmán, Dan Jurafsky +1
Hungarian is often referred to as a discourse-configurational language, since the structural position of constituents is determined by their logical function (topic or comment) rat…
GoEmotions: A Dataset of Fine-Grained Emotions
Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko +3
Understanding emotion expressed in language has a wide range of applications, from building empathetic chatbots to detecting harmful online behavior. Advancement in this area can b…
Analyzing Polarization in Social Media: Method and Application to Tweets on 21 Mass Shootings
Dorottya Demszky, Nikhil Garg, Rob Voigt +4
We provide an NLP framework to uncover four linguistic dimensions of political polarization in social media: topic choice, framing, affect and illocutionary force. We quantify thes…
Transforming Question Answering Datasets Into Natural Language Inference Datasets
Dorottya Demszky, Kelvin Guu, Percy Liang
Existing datasets for natural language inference (NLI) have propelled research on language understanding. We propose a new method for automatically deriving NLI datasets from the g…