8 papers
DRIP-R: A Benchmark for Decision-Making and Reasoning Under Real-World Policy Ambiguity in the Retail Domain
Hsuvas Borkakoty, Sebastian Pohl, Cheng Wang +2
LLM-based agents are increasingly deployed for routine but consequential tasks in real-world domains, where their behavior is governed by inherently ambiguous domain policies that…
Where Knowledge Collides: A Mechanistic Study of Intra-Memory Knowledge Conflict in Language Models
Minh Vu Pham, Hsuvas Borkakoty, Yufang Hou
In language models (LMs), intra-memory knowledge conflict arises when inconsistent information about the same subject is encoded within the model's parametric knowledge. Prior work…
Wikipedia is Not a Dictionary, Delete! Text Classification as a Proxy for Analysing Wiki Deletion Discussions
Hsuvas Borkakoty, Luis Espinosa-Anke
Automated content moderation for collaborative knowledge hubs like Wikipedia or Wikidata is an important yet challenging task due to multiple factors. In this paper, we construct a…
Automatic Extraction of Metaphoric Analogies from Literary Texts: Task Formulation, Dataset Construction, and Evaluation
Joanne Boisson, Zara Siddique, Hsuvas Borkakoty +3
Extracting metaphors and analogies from free text requires high-level reasoning abilities such as abstraction and language understanding. Our study focuses on the extraction of the…
WiDe-analysis: Enabling One-click Content Moderation Analysis on Wikipedia's Articles for Deletion
Hsuvas Borkakoty, Luis Espinosa-Anke
Content moderation in online platforms is crucial for ensuring activity therein adheres to existing policies, especially as these platforms grow. NLP research in this area has typi…
CHEW: A Dataset of CHanging Events in Wikipedia
Hsuvas Borkakoty, Luis Espinosa-Anke
We introduce CHEW, a novel dataset of changing events in Wikipedia expressed in naturally occurring text. We use CHEW for probing LLMs for their timeline understanding of Wikipedia…