5 papers
When Do Hallucinations Arise? A Graph Perspective on the Evolution of Path Reuse and Path Compression
Xinnan Dai, Kai Yang, Cheng Luo +3
Reasoning hallucinations in large language models (LLMs) often appear as fluent yet unsupported conclusions that violate either the given context or underlying factual knowledge. A…
Fix Before Search: Benchmarking Agentic Query Visual Pre-processing in Multimodal Retrieval-augmented Generation
Jiankun Zhang, Shenglai Zeng, Kai Guo +4
Multimodal Retrieval-Augmented Generation (MRAG) has emerged as a key paradigm for grounding MLLMs with external knowledge. While query pre-processing (e.g., rewriting) is standard…
GraphGhost: Tracing Structures Behind Large Language Models
Xinnan Dai, Xianxuan Long, Chung-Hsiang Lo +4
Large Language Models (LLMs) exhibit strong reasoning capabilities on structured tasks, yet the internal mechanisms underlying such behaviors remain poorly understood. Existing int…
Uncovering Graph Reasoning in Decoder-only Transformers with Circuit Tracing
Xinnan Dai, Chung-Hsiang Lo, Kai Guo +3
Transformer-based LLMs demonstrate strong performance on graph reasoning tasks, yet their internal mechanisms remain underexplored. To uncover these reasoning process mechanisms in…
From Sequence to Structure: Uncovering Substructure Reasoning in Transformers
Xinnan Dai, Kai Yang, Jay Revolinsky +4
Recent studies suggest that large language models (LLMs) possess the capability to solve graph reasoning tasks. Notably, even when graph structures are embedded within textual desc…