11 papers
GUIDE: Guiding Internal Evidence with Language Instructions
Soyeon Caren Han, Hyunsuk Chung, Jinwoo Kim +2
Large multimodal models follow instructions about what to generate, but not necessarily about what evidence to rely on. Hence, models may continue to depend on shortcut-associated…
When Correct Decisions Hide Internal Stress: Decision-State Probing in Multimodal Language Models
Haoran Zhao, Soyeon Caren Han, Eduard Hovy
Multimodal language models are typically evaluated through external behavior: selecting the correct image--text match, rejecting unsupported captions, or answering visual queries c…
Inclusion-of-Thoughts: Mitigating Preference Instability via Purifying the Decision Space
Mohammad Reza Ghasemi Madani, Soyeon Caren Han, Shuo Yang +1
Multiple-choice questions (MCQs) are widely used to evaluate large language models (LLMs). However, LLMs remain vulnerable to the presence of plausible distractors. This often dive…
ToolTree: Efficient LLM Agent Tool Planning via Dual-Feedback Monte Carlo Tree Search and Bidirectional Pruning
Shuo Yang, Soyeon Caren Han, Yihao Ding +2
Large Language Model (LLM) agents are increasingly applied to complex, multi-step tasks that require interaction with diverse external tools across various domains. However, curren…
BRIDGE: Benchmark for multi-hop Reasoning In long multimodal Documents with Grounded Evidence
Biao Xiang, Soyeon Caren Han, Yihao Ding
Multi-hop question answering (QA) is widely used to evaluate the reasoning capabilities of large language models, yet most benchmarks focus on final answer correctness and overlook…
EvoTool: Self-Evolving Tool-Use Policy Optimization in LLM Agents via Blame-Aware Mutation and Diversity-Aware Selection
Shuo Yang, Soyeon Caren Han, Xueqi Ma +3
LLM-based agents depend on effective tool-use policies to solve complex tasks, yet optimizing these policies remains challenging due to delayed supervision and the difficulty of cr…
When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning
Muku Akasaka, Soyeon Caren Han
Visual spatial reasoning (VSR) remains challenging for modern vision-language models (VLMs), despite advances in multimodal architectures. A common strategy is to inject additional…
Diagnosing Causal Reasoning in Vision-Language Models via Structured Relevance Graphs
Dhita Putri Pratama, Soyeon Caren Han, Yihao Ding
Large Vision-Language Models (LVLMs) achieve strong performance on visual question answering benchmarks, yet often rely on spurious correlations rather than genuine causal reasonin…
Physics-based phenomenological characterization of cross-modal bias in multimodal models
Hyeongmo Kim, Sohyun Kang, Yerin Choi +5
The term 'algorithmic fairness' is used to evaluate whether AI models operate fairly in both comparative (where fairness is understood as formal equality, such as "treat like cases…
When Is Rank-1 Enough? Geometry-Guided Initialization for Parameter-Efficient Fine-Tuning
Haoran Zhao, Soyeon Caren Han, Eduard Hovy
Parameter-efficient fine-tuning (PEFT) is a standard way to adapt multimodal large language models, yet extremely low-rank settings -- especially rank-1 LoRA -- are often unstable.…
HierCon: Hierarchical Contrastive Attention for Audio Deepfake Detection
Zhili Nicholas Liang, Soyeon Caren Han, Qizhou Wang +1
Audio deepfakes generated by modern TTS and voice conversion systems are increasingly difficult to distinguish from real speech, raising serious risks for security and online trust…