2 citations · 4 across the 9 of their papers we have counts for
10 papers
PyMETA: Evaluating Student Code Diagnosis on and Beyond the First Execution Error
Chuyue Li, Ziqi Tang, Jingyi Wang +3
Large language models can diagnose a student program from its code, problem statement, and reference solution. Evaluating this ability requires a clear definition of what counts as…
WhoSaidIt: Human-LLM Collaborative Annotation for Text-Based Multilingual Speaker-Attribute Classification
Lingyu Gao, Will Monroe, David Smith +2
Annotating speaker attributes from text is inherently ambiguous, particularly in multilingual settings where demographic and social cues are implicit and culturally variable. We pr…
Harnessing the Intrinsic Knowledge of Pretrained Language Models for Challenging Text Classification Settings
Lingyu Gao
Text classification is crucial for applications such as sentiment analysis and toxic text filtering, but it still faces challenges due to the complexity and ambiguity of natural la…
Ambiguity-Aware In-Context Learning with Large Language Models
Lingyu Gao, Aditi Chaudhary, Krishna Srinivasan +3
In-context learning (ICL) i.e. showing LLMs only a few task-specific demonstrations has led to downstream gains with no task-specific fine-tuning required. However, LLMs are sensit…
ToMChallenges: A Principle-Guided Dataset and Diverse Evaluation Tasks for Exploring Theory of Mind
Xiaomeng Ma, Lingyu Gao, Qihui Xu
Theory of Mind (ToM), the capacity to comprehend the mental states of distinct individuals, is essential for numerous practical applications. With the development of large language…
The Benefits of Label-Description Training for Zero-Shot Text Classification
Lingyu Gao, Debanjan Ghosh, Kevin Gimpel
Pretrained language models have improved zero-shot text classification by allowing the transfer of semantic knowledge from the training data in order to classify among specific lab…
Evaluating Transformer Models and Human Behaviors on Chinese Character Naming
Xiaomeng Ma, Lingyu Gao
Neural network models have been proposed to explain the grapheme-phoneme mapping process in humans for many alphabet languages. These models not only successfully learned the corre…
How do we get there? Evaluating transformer neural networks as cognitive models for English past tense inflection
Xiaomeng Ma, Lingyu Gao
There is an ongoing debate on whether neural networks can grasp the quasi-regularities in languages like humans. In a typical quasi-regularity task, English past tense inflections,…
"What makes a question inquisitive?" A Study on Type-Controlled Inquisitive Question Generation
Lingyu Gao, Debanjan Ghosh, Kevin Gimpel
We propose a type-controlled framework for inquisitive question generation. We annotate an inquisitive question dataset with question types, train question type classifiers, and fi…
A Cross-Task Analysis of Text Span Representations
Shubham Toshniwal, Haoyue Shi, Bowen Shi +3
Many natural language processing (NLP) tasks involve reasoning with textual spans, including question answering, entity recognition, and coreference resolution. While extensive res…