4 papers · 1 filter
Evaluating Context Compaction on the Questions That Come Later
Junwei Yao, Yongxi Zhou, Zike Yang +4
Language-model agents compact their context for the question at hand, but the compacted context must also serve questions that come later. We evaluate compaction on these later que…
Pigeonholing: how bad prompts hurt models, causing collapse and mistakes
Hyunji Nam, Keertana Chidambaram, Dorottya Demszky +1
While in-context learning is generally shown to be effective in Large Language Models (LLMs), bad contexts can cause performance degradation and mode collapse, a phenomenon we call…
IDEAlign: Comparing Ideas of Large Language Models to Domain Expert
Hyunji Nam, Lucia Langlois, James Malamut +2
Large language models (LLMs) are increasingly used to produce open-ended, interpretive annotations, yet there is no validated, scalable measure of idea-level similarity to expert a…
EduCoder: An Open-Source Annotation System for Education Transcript Data
Saad Ashraf, James Malamut, Vishal Kumar +7
We introduce EduCoder, a domain-specialized tool designed to support utterance-level annotation of educational dialogue. While general-purpose text annotation tools for NLP and qua…