7 papers
ClaimFlow: Tracing the Evolution of Scientific Claims in NLP
Aniket Pramanick, Yufang Hou, Saif M. Mohammad +1
Scientific papers advance that later work supports, extends, or sometimes refutes. Yet existing methods for citation and claim analysis capture only fragments of…
Holmes: A Benchmark to Assess the Linguistic Competence of Language Models
Andreas Waldis, Yotam Perlitz, Leshem Choshen +2
We introduce Holmes, a new benchmark designed to assess language models (LMs) linguistic competence - their unconscious understanding of linguistic phenomena. Specifically, we use…
DRIP-R: A Benchmark for Decision-Making and Reasoning Under Real-World Policy Ambiguity in the Retail Domain
Hsuvas Borkakoty, Sebastian Pohl, Cheng Wang +2
LLM-based agents are increasingly deployed for routine but consequential tasks in real-world domains, where their behavior is governed by inherently ambiguous domain policies that…
TRACE: Tourism Recommendation with Accountable Citation Evidence
Zixu Zhao, Sijin Wang, Yu Hou +6
Tourism is a high-stakes setting for conversational recommender systems (CRS): a plausible-sounding suggestion can waste real money and trip time once a traveler acts on it. Existi…
A Multi-View Media Profiling Suite: Resources, Evaluation, and Analysis
Muhammad Arslan Manzoor, Dilshod Azizov, Daniil Orel +4
News outlets shape public opinion at a scale that makes automated detection of political bias and factuality essential. However, the field still lacks unified resources, comprehens…
The Nature of NLP: Analyzing Contributions in NLP Papers
Aniket Pramanick, Yufang Hou, Saif M. Mohammad +1
Natural Language Processing (NLP) is an established and dynamic field. Despite this, what constitutes NLP research remains debated. In this work, we address the question by quantit…