9 papers · 1 filter
SagaQA: A Multi-hop Reasoning Benchmark for Long-form Narrative Understanding in TV Series
Galann Pennec, Zhengyuan Liu, Nicholas Asher +2
We introduce SagaQA, a long-form video benchmark for multi-hop reasoning over full-length TV series. Existing video reasoning benchmarks often emphasize local understanding of adja…
The Deliberative Illusion: Diagnosing Factual Attrition and Stance Homogenization in Multi-Agent LLM Deliberation
Herun Wan, Jiaying Wu, Minnan Luo +4
Multi-agent LLM systems often treat consensus as evidence of successful interaction. For deliberative problems, however, reliability depends on whether agents preserve the facts an…
MeMo: Memory as a Model
Ryan Wei Heng Quek, Sanghyuk Lee, Alfred Wei Lun Leong +6
Large language models (LLMs) achieve strong performance across a wide range of tasks, but remain frozen after pretraining until subsequent updates. Many real-world applications req…
What Makes a Good Natural Language Prompt?
Do Xuan Long, Duy Dinh, Ngoc-Hai Nguyen +4
As large language models (LLMs) have progressed towards more human-like and human--AI communications have become prevalent, prompting has emerged as a decisive component. However,…
Beyond In-Context Learning: Aligning Long-form Generation of Large Language Models via Task-Inherent Attribute Guidelines
Do Xuan Long, Duong Ngoc Yen, Do Xuan Trong +5
In-context learning (ICL) is an important yet not fully understood ability of pre-trained large language models (LLMs). It can greatly enhance task performance using a few examples…
LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMs
Do Xuan Long, Hai Nguyen Ngoc, Tiviatis Sim +5
We present the first systematic evaluation examining format bias in performance of large language models (LLMs). Our approach distinguishes between two categories of an evaluation…