565 citations · 881 across the 17 of their papers we have counts for
19 papers
Choice-75: A Dataset on Decision Branching in Script Learning
Zhaoyi Joey Hou, Li Zhang, Chris Callison-Burch
Script learning studies how stereotypical events unfold, enabling machines to reason about narratives with implicit information. Previous works mostly consider a script as a linear…
OpenPI2.0: An Improved Dataset for Entity Tracking in Texts
Li Zhang, Hainiu Xu, Abhinav Kommula +2
Much text describes a changing world (e.g., procedures, stories, newswires), and understanding them requires tracking how entities change. An earlier dataset, OpenPI, provided crow…
Exploring the Curious Case of Code Prompts
Li Zhang, Liam Dugan, Hainiu Xu +1
Recent work has shown that prompting language models with code-like representations of natural language leads to performance improvements on structured reasoning tasks. However, su…
Human-in-the-Loop Schema Induction
Tianyi Zhang, Isaac Tham, Zhaoyi Hou +12
Schema induction builds a graph representation explaining how events unfold in a scenario. Existing approaches have been based on information retrieval (IR) and information extract…
Causal Reasoning of Entities and Events in Procedural Texts
Li Zhang, Hainiu Xu, Yue Yang +4
Entities and events are crucial to natural language reasoning and common in procedural texts. Existing work has focused either exclusively on entity state tracking (e.g., whether a…
Language Models are Drummers: Drum Composition with Natural Language Pre-Training
Li Zhang, Chris Callison-Burch
Automatic music generation with artificial intelligence typically requires a large amount of data which is hard to obtain for many less common genres and musical instruments. To ta…
Faithful Chain-of-Thought Reasoning
Qing Lyu, Shreya Havaldar, Adam Stein +5
While Chain-of-Thought (CoT) prompting boosts Language Models' (LM) performance on a gamut of complex reasoning tasks, the generated reasoning chain does not necessarily reflect ho…
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code
Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran +74
Evaluation in machine learning is usually informed by past choices, for example which datasets or metrics to use. This standardization enables the comparison on equal footing using…
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448
Language models demonstrate both quantitative improvement and new qualitative capabilities with increasing scale. Despite their potentially transformative impact, these new capabil…
Reasoning about Procedures with Natural Language Processing: A Tutorial
Li Zhang
This tutorial provides a comprehensive and in-depth view of the research on procedures, primarily in Natural Language Processing. A procedure is a sequence of steps intended to ach…
Show Me More Details: Discovering Hierarchies of Procedures from Semi-structured Web Data
Shuyan Zhou, Li Zhang, Yue Yang +4
Procedures are inherently hierarchical. To "make videos", one may need to "purchase a camera", which in turn may require one to "set a budget". While such hierarchical knowledge is…
Is "My Favorite New Movie" My Favorite Movie? Probing the Understanding of Recursive Noun Phrases
Qing Lyu, Hua Zheng, Daoxin Li +3
Recursive noun phrases (NPs) have interesting semantic properties. For example, "my favorite new movie" is not necessarily my favorite movie, whereas "my new favorite movie" is. Th…
Goal-Oriented Script Construction
Qing Lyu, Li Zhang, Chris Callison-Burch
The knowledge of scripts, common chains of events in stereotypical scenarios, is a valuable asset for task-oriented natural language understanding systems. We propose the Goal-Orie…
Visual Goal-Step Inference using wikiHow
Yue Yang, Artemis Panagopoulou, Qing Lyu +3
Understanding what sequence of steps are needed to complete a goal can help artificial intelligence systems reason about human activities. Past work in NLP has examined the task of…
Small but Mighty: New Benchmarks for Split and Rephrase
Li Zhang, Huaiyu Zhu, Siddhartha Brahma +1
Split and Rephrase is a text simplification task of rewriting a complex sentence into simpler ones. As a relatively new task, it is paramount to ensure the soundness of its evaluat…
Reasoning about Goals, Steps, and Temporal Ordering with WikiHow
Li Zhang, Qing Lyu, Chris Callison-Burch
We propose a suite of reasoning tasks on two types of relations between procedural events: goal-step relations ("learn poses" is a step in the larger goal of "doing yoga") and step…
Intent Detection with WikiHow
Li Zhang, Qing Lyu, Chris Callison-Burch
Modern task-oriented dialog systems need to reliably understand users' intents. Intent detection is most challenging when moving to new domains or new languages, since there is lit…
Improving Text-to-SQL Evaluation Methodology
Catherine Finegan-Dollak, Jonathan K. Kummerfeld, Li Zhang +4
To be informative, an evaluation must measure how well systems generalize to realistic unseen data. We identify limitations of and propose improvements to current evaluations of te…
Direct Network Transfer: Transfer Learning of Sentence Embeddings for Semantic Similarity
Li Zhang, Steven R. Wilson, Rada Mihalcea
Sentence encoders, which produce sentence embeddings using neural networks, are typically evaluated by how well they transfer to downstream tasks. This includes semantic similarity…