1 citations · 1 across the 11 of their papers we have counts for
14 papers
Back Up Before You Regret: Measuring Backup Awareness and Automating Backup in Coding Agents
Xu Li, Simon Yu, Minzhou Pan +4
Coding agents now complete tasks autonomously, modifying real-world state such as artifacts, software environments, and configurations along the way. Yet they show limited backup a…
VerifierBench: Benchmarking the Generation of Executable Verifiers for LLM Agents
Jianing Yin, Ziyi Chen, Hao Wang +4
Agents need reliable ways to validate their outputs, both when completing tasks and when filtering synthetic training data. We study verifier generation: from the instruction and e…
Agents' Last Exam
Yiyou Sun, Xinyang Han, Weichen Zhang +378
Recent AI systems have achieved strong results on a wide range of benchmarks, yet these gains have not translated into economically meaningful deployment across many professional d…
The Long-Horizon Task Mirage? Diagnosing Where and Why Agentic Systems Break
Xinyu Jessica Wang, Haoyue Bai, Yiyou Sun +7
Large language model (LLM) agents perform strongly on short- and mid-horizon tasks, but often break down on long-horizon tasks that require extended, interdependent action sequence…
Strategy Executability in Mathematical Reasoning: Leveraging Human-Model Differences for Effective Guidance
Weida Liang, Yiyou Sun, Shuyuan Nan +3
Example-based guidance is widely used to improve mathematical reasoning at inference time, yet its effectiveness is highly unstable across problems and models-even when the guidanc…
Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents
Xu Li, Simon Yu, Minzhou Pan +5
LLM-based agents are becoming increasingly capable, yet their safety lags behind. This creates a gap between what agents can do and should do. This gap widens as agents engage in m…
How and Why LLMs Generalize: A Fine-Grained Analysis of LLM Reasoning from Cognitive Behaviors to Low-Level Patterns
Haoyue Bai, Yiyou Sun, Wenjie Hu +5
Large Language Models (LLMs) display strikingly different generalization behaviors: supervised fine-tuning (SFT) often narrows capability, whereas reinforcement-learning (RL) tunin…
RL Grokking Recipe: How Does RL Unlock and Transfer New Algorithms in LLMs?
Yiyou Sun, Yuhan Cao, Pohao Huang +4
It remains an open question whether LLMs can acquire or generalize genuinely new reasoning strategies, beyond the sharpened skills encoded in their parameters during pre-training o…
Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thought
Jiachen Zhao, Yiyou Sun, Weiyan Shi +1
Large language models can generate long chain-of-thought (CoT) reasoning, yet prior work suggests that CoT can be post-hoc rationalization rather than a faithful reflection of the…
MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Them
Weichen Zhang, Yiyou Sun, Pohao Huang +3
Hallucinations pose critical risks for large language model (LLM)-based agents, often manifesting as hallucinative actions resulting from fabricated or misinterpreted information w…
OMEGA: Can LLMs Reason Outside the Box in Math? Evaluating Exploratory, Compositional, and Transformative Generalization
Yiyou Sun, Shawn Hu, Georgia Zhou +4
Recent large-scale language models (LLMs) with long Chain-of-Thought reasoning-such as DeepSeek-R1-have achieved impressive results on Olympiad-level mathematics benchmarks. Howeve…
Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations
Yiyou Sun, Yu Gai, Lijie Chen +3
Large language models (LLMs) frequently generate hallucinations-content that deviates from factual accuracy or provided context-posing challenges for diagnosis due to the complex i…
Climbing the Ladder of Reasoning: What LLMs Can-and Still Can't-Solve after SFT?
Yiyou Sun, Georgia Zhou, Haoyue Bai +4
Recent supervised fine-tuning (SFT) approaches have significantly improved language models' performance on mathematical reasoning tasks, even when models are trained at a small sca…
Can LLMs Ask Good Questions?
Yueheng Zhang, Xiaoyuan Liu, Yiyou Sun +5
We evaluate questions generated by large language models (LLMs) from context, comparing them to human-authored questions across six dimensions: question type, question length, cont…