11 papers
TRILOGUE: A Trilingual Spoken Dialogue Fact-Checking Benchmark with Evidence and Paired Audio
Chaewan Chun, Meruyert Aristombayeva, Jiyoung Choi +3
Modern misinformation is often heard before it is read, yet fact-checking systems are still evaluated mainly on clean written claims. Spoken dialogue remains different even when sy…
Lost in Speech: Trilingual Spoken Hallucination Detection Across Audio and Transcripts
Meruyert Aristombayeva, Jason S. Lucas, Chaewan Chun +1
While text-based hallucination detection is well studied, reference-free detection of factual alterations in speech remains underexplored, especially for low-resource languages. Ou…
Lost in Context Compaction: When LLM Systems Drop Side Constraints
Zhiqi Wang, Yichi Zhang, Dongwon Lee +2
When the context window is under pressure, LLM systems compact prior context to continue ongoing tasks. We study Side Constraints (SCs), user-issued instructions such as "do not de…
Context-Aware Multimodal Claim Verification in Spoken Dialogues
Chaewan Chun, Delvin Ce Zhang, Dongwon Lee
Spoken factual claims often occur within multi-turn conversations, where surrounding dialogue can provide context unavailable from the claim alone. Yet most fact-checking research…
AOR-Bench: Do Large Audio Language Models Over-Refuse Pseudo-Harmful Queries?
Jiaxi Yang, Chaewan Chun, Jason Lucas +2
Large Audio Language Models (LALMs) have demonstrated strong performance across a wide range of audio tasks. As they are increasingly deployed in real-world applications, ensuring…
Moltbook Moderation: Uncovering Hidden Intent Through Multi-Turn Dialogue
Ali Al-Lawati, Nafis Tripto, Abolfazl Ansari +3
The emergence of multi-agent systems introduces novel moderation challenges that extend beyond content filtering. Agents with malicious intent may contribute harmful content that a…
M2-Verify: A Large-Scale Multidomain Benchmark for Checking Multimodal Claim Consistency
Abolfazl Ansari, Delvin Ce Zhang, Zhuoyang Zou +2
Evaluating scientific arguments requires assessing the strict consistency between a claim and its underlying multimodal evidence. However, existing benchmarks lack the scale, domai…
DIAGPaper: Diagnosing Valid and Specific Weaknesses in Scientific Papers via Multi-Agent Reasoning
Zhuoyang Zou, Abolfazl Ansari, Delvin Ce Zhang +2
Paper weakness identification using single-agent or multi-agent LLMs has attracted increasing attention, yet existing approaches exhibit key limitations. Many multi-agent systems s…
Unmasking Fake Careers: Detecting Machine-Generated Career Trajectories via Multi-layer Heterogeneous Graphs
Michiharu Yamashita, Thanh Tran, Delvin Ce Zhang +1
The rapid advancement of Large Language Models (LLMs) has enabled the generation of highly realistic synthetic data. We identify a new vulnerability, LLMs generating convincing car…
MAD: A Benchmark for Multi-Turn Audio Dialogue Fact-Checking
Chaewan Chun, Lysandre Terrisse, Delvin Ce Zhang +1
Despite the growing popularity of audio platforms, fact-checking spoken content remains significantly underdeveloped. Misinformation in speech often unfolds across multi-turn dialo…
Could AI Trace and Explain the Origins of AI-Generated Images and Text?
Hongchao Fang, Yixin Liu, Jiangshu Du +7
AI-generated content is becoming increasingly prevalent in the real world, leading to serious ethical and societal concerns. For instance, adversaries might exploit large multimoda…