5 citations · 19 across the 10 of their papers we have counts for
10 papers
Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents
Minhua Lin, Juncheng Wu, Zijun Wang +14
LLM agents are increasingly deployed as systems built around editable external harnesses, including prompts, skills, memories and tools, that shape task execution without changing…
Trajectory2Task: Training Robust Tool-Calling Agents with Synthesized Yet Verifiable Data for Complex User Intents
Ziyi Wang, Yuxuan Lu, Yimeng Zhang +12
Tool-calling agents are increasingly deployed in real-world customer-facing workflows. Yet most studies on tool-calling agents focus on idealized settings with general, fixed, and…
Few-Shot Character Understanding in Movies as an Assessment to Meta-Learning of Theory-of-Mind
Mo Yu, Qiujing Wang, Shunchi Zhang +8
When reading a story, humans can quickly understand new fictional characters with a few observations, mainly by drawing analogies to fictional and real people they already know. Th…
MBTI Personality Prediction for Fictional Characters Using Movie Scripts
Yisi Sang, Xiangyang Mou, Mo Yu +3
An NLP model that understands stories should be able to understand the characters in them. To support the development of neural models for this purpose, we construct a benchmark, S…
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code
Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran +74
Evaluation in machine learning is usually informed by past choices, for example which datasets or metrics to use. This standardization enables the comparison on equal footing using…
A Survey of Machine Narrative Reading Comprehension Assessments
Yisi Sang, Xiangyang Mou, Jing Li +2
As the body of research on machine narrative comprehension grows, there is a critical need for consideration of performance assessment strategies as well as the depth and scope of…
TVShowGuess: Character Comprehension in Stories as Speaker Guessing
Yisi Sang, Xiangyang Mou, Mo Yu +3
We propose a new task for assessing machines' skills of understanding fictional characters in narrative stories. The task, TVShowGuess, builds on the scripts of TV series and takes…
Fantastic Questions and Where to Find Them: FairytaleQA -- An Authentic Dataset for Narrative Comprehension
Ying Xu, Dakuo Wang, Mo Yu +15
Question answering (QA) is a fundamental means to facilitate assessment and training of narrative comprehension skills for both machines and young children, yet there is scarcity o…
The Origin and Value of Disagreement Among Data Labelers: A Case Study of the Individual Difference in Hate Speech Annotation
Yisi Sang, Jeffrey Stanton
Human annotated data is the cornerstone of today's artificial intelligence efforts, yet data labeling processes can be complicated and expensive, especially when human labelers dis…
Malicious Selling Strategies in E-Commerce Livestream: A Case Study of Alibaba's Taobao and ByteDance's TikTok
Qunfang Wu, Yisi Sang, Dakuo Wang +1
Due to the limitations imposed by the COVID-19 pandemic, many users have shifted their shopping patterns from offline to online. Livestream shopping has become popular as one of th…