4 papers
Measuring Semantic Progress in Multi-turn Dialogue via Information Gain
Paul He, Shiva Kasiviswanathan, Dominik Janzing
Evaluating multi-turn dialogue is challenging because quality emerges across turns rather than within individual responses. We focus on a key dimension of information-seeking dialo…
Uncovering Hidden Correctness in LLM Causal Reasoning via Symbolic Verification
Paul He, Yinya Huang, Mrinmaya Sachan +1
Large language models (LLMs) are increasingly being applied to tasks that involve causal reasoning. However, current benchmarks often rely on string matching or surface-level metri…
Foundations of Global Consistency Checking with Noisy LLM Oracles
Paul He, Elke Kirschbaum, Shiva Kasiviswanathan
Ensuring that collections of natural-language facts are globally consistent is essential for tasks such as fact-checking, summarization, and knowledge base construction. While Larg…
Pointwise Mutual Information as a Performance Gauge for Retrieval-Augmented Generation
Tianyu Liu, Jirui Qi, Paul He +3
Recent work suggests that large language models enhanced with retrieval-augmented generation are easily influenced by the order, in which the retrieved documents are presented to t…