4 papers · 1 filter
Tracing How Annotators Think: Augmenting Preference Judgments with Reading Processes
Karin de Langis, William Walker, Khanh Chi Le +1
We propose an annotation approach that captures not only labels but also the reading process underlying annotators' decisions, e.g., what parts of the text they focus on, re-read o…
Strong Memory, Weak Control: An Empirical Study of Executive Functioning in LLMs
Karin de Langis, Jong Inn Park, Bin Hu +5
Working memory, or the ability to hold and manipulate information in the mind, is a critical component of human intelligence and executive functioning. It is correlated with perfor…
LawFlow: Collecting and Simulating Lawyers' Thought Processes on Business Formation Case Studies
Debarati Das, Khanh Chi Le, Ritik Sachin Parkar +8
Legal practitioners, particularly those early in their careers, face complex, high-stakes tasks that require adaptive, context-sensitive reasoning. While AI holds promise in suppor…
How LLMs Comprehend Temporal Meaning in Narratives: A Case Study in Cognitive Evaluation of LLMs
Karin de Langis, Jong Inn Park, Andreas Schramm +5
Large language models (LLMs) exhibit increasingly sophisticated linguistic capabilities, yet the extent to which these behaviors reflect human-like cognition versus advanced patter…