7 papers
Search Engines Never Say No: How Frozen Agents React When the Retrieval Tool Refuses
Ramraj Chandradevan, Sayontan Ghosh, Vinoth Selvendran
A search tool never says no: it returns its top-k passages even when the index holds no answer, so the agent sees irrelevant text instead of a miss signal. We ask what frozen searc…
Handling Open-Vocabulary Constructs in Formalizing Specifications: Retrieval-Augmented Parsing with Expert Knowledge
Mohammad Saqib Hasan, Sayontan Ghosh, Dhruv Verma +4
We study the problem of Open-Vocabulary Constructs(OVCs) -- ones not known beforehand -- in the context of converting natural language (NL) specifications into formal languages (e.…
: A Dataset for ynamic nformation nd ental modeling f umeric iscussions
Sayontan Ghosh, Mahnaz Koupaee, Yash Kumar Lal +4
Understanding multiparty conversations demands robust Theory of Mind (ToM) capabilities, including the ability to track dynamic information, manage knowledge asymmetries, and disti…
SandboxAQ's submission to MRL 2024 Shared Task on Multi-lingual Multi-task Information Retrieval
Isidora Chara Tourni, Sayontan Ghosh, Brenda Miao +1
This paper explores the problems of Question Answering (QA) and Named Entity Recognition (NER) in five diverse languages. We tested five Large Language Models with various promptin…
POQue: Asking Participant-specific Outcome Questions for a Deeper Understanding of Complex Events
Sai Vallurupalli, Sayontan Ghosh, Katrin Erk +2
Knowledge about outcomes is critical for complex event understanding but is hard to acquire. We show that by pre-identifying a participant in a complex event, crowd workers are abl…
Text-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipation
Sayontan Ghosh, Tanvi Aggarwal, Minh Hoai +1
Anticipating future actions in a video is useful for many autonomous and assistive technologies. Most prior action anticipation work treat this as a vision modality problem, where…
PASTA: A Dataset for Modeling Participant States in Narratives
Sayontan Ghosh, Mahnaz Koupaee, Isabella Chen +3
The events in a narrative are understood as a coherent whole via the underlying states of their participants. Often, these participant states are not explicitly mentioned, instead…