8 papers
When is Your LLM Steerable?
Chenrui Fan, Yize Cheng, Ming Li +2
Activation steering offers a lightweight approach to control language models' behavior at inference time, but whether it succeeds or fails heavily depends on the prompt, concept, m…
V-REX: Benchmarking Exploratory Visual Reasoning via Chain-of-Questions
Chenrui Fan, Yijun Liang, Shweta Bhardwaj +3
While many vision-language models (VLMs) are developed to answer well-defined, straightforward questions with highly specified targets, as in most benchmarks, they often struggle i…
Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Use
Yize Cheng, Chenrui Fan, Mahdi JafariRaviz +2
Large language models (LLMs) increasingly act as autonomous agents that must decide when to answer directly vs. when to invoke external tools. Prior work studying adaptive tool use…
Your LLM Agents are Temporally Blind: The Misalignment Between Tool Use Decisions and Human Time Perception
Yize Cheng, Arshia Soltani Moakhar, Chenrui Fan +5
Large language model (LLM) agents are increasingly used to interact with and execute tasks in dynamic environments. However, a critical yet overlooked limitation of these agents is…
Grokking in LLM Pretraining? Monitor Memorization-to-Generalization without Test
Ziyue Li, Chenrui Fan, Tianyi Zhou
This paper presents the first study of grokking in practical LLM pretraining. Specifically, we investigate when an LLM memorizes the training data, when its generalization on downs…
ColorBench: Can VLMs See and Understand the Colorful World? A Comprehensive Benchmark for Color Perception, Reasoning, and Robustness
Yijun Liang, Ming Li, Chenrui Fan +7
Color plays an important role in human perception and usually provides critical clues in visual reasoning. However, it is unclear whether and how vision-language models (VLMs) can…