6 papers
ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Research
Wanghan Xu, Shuo Li, Tianlin Ye +48
AI coding agents are increasingly used for scientific work, but their end-to-end autonomous research capability remains difficult to verify. We present ResearchClawBench, a benchma…
Dockerless: Environment-Free Program Verifier for Coding Agents
Wenhao Zeng, Yuling Shi, Xiaodong Gu +10
Program verifiers play a central role in training coding agents, including selecting trajectories for supervised fine-tuning (SFT) and providing rewards for reinforcement learning…
In Line with Context: Repository-Level Code Generation via Context Inlining
Chao Hu, Wenhao Zeng, Yuling Shi +2
Repository-level code generation has attracted growing attention in recent years. Unlike function-level code generation, it requires the model to understand the entire repository,…
GlimpRouter: Efficient Collaborative Inference by Glimpsing One Token of Thoughts
Wenhao Zeng, Xuteng Zhang, Yuling Shi +4
Large Reasoning Models (LRMs) achieve remarkable performance by explicitly generating multi-step chains of thought, but this capability incurs substantial inference latency and com…
ShredBench: Evaluating the Semantic Reasoning Capabilities of Multimodal LLMs in Document Reconstruction
Zichun Guo, Yuling Shi, Wenhao Zeng +6
Multimodal Large Language Models (MLLMs) have achieved remarkable performance in Visually Rich Document Understanding (VRDU) tasks, but their capabilities are mainly evaluated on p…
Pruning the Unsurprising: Efficient LLM Reasoning via First-Token Surprisal
Wenhao Zeng, Yaoning Wang, Chao Hu +4
Large Reasoning Models (LRMs) have demonstrated remarkable capabilities by scaling up the length of Chain-of-Thought (CoT). However, excessively long reasoning traces pose substant…