20 papers
EvalBuilder: Can LLMs Build Efficient Benchmarks?
Huaisheng Zhu, Hangfan Zhang, Teng Xiao +5
Large language model (LLM) agents are increasingly used for long-horizon research and engineering, yet evaluating these capabilities remains expensive, time-consuming, and heavily…
Rethinking the Evaluation of Harness Evolution for Agents
Yike Wang, Huaisheng Zhu, Zhengyu Hu +8
Harness evolution is an iterative search procedure that repeatedly evaluates and revises candidate harnesses used for LLM agents using task feedback. We revisit the evaluation of s…
Beyond Final Scores: A Systematic Evaluation of Agents for Long-Horizon AI Research and Development
Yiwei Li, Wanli Yang, Hexiang Tan +10
Autonomous agents are increasingly capable of improving models, systems, and other technical artifacts through long-horizon experimentation. To understand the current state of this…
HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses
Luan Zhang, Ruochen Zhou, Dandan Song +9
Harness design plays a critical role in agent performance by shaping how large language models (LLMs) perceive, reason over, and act within executable environments. Recent work has…
Distilling Temporal Search and Reasoning: Evolving LLMs for Future Prediction via Harness-Assisted Efficient Data Synthesis
Wanxu Cai, Zhengyu Chen, Huaisheng Zhu +3
Future event prediction carries broad social impact yet remains challenging. SOTA approaches augment LLMs with external agent frameworks whose predictive capability vanishes once t…
Co-Harness: Co-Evolving Harnesses and Model Weights for LLM Agents
Zhengyu Chen, Teng Xiao, Huaisheng Zhu +3
Post-training agents for automated AI research requires optimizing not only model parameters, but also the runtime harness that shapes how research trajectories are generated, eval…
PruneTIR: Inference-Time Tool Call Pruning for Effective yet Efficient Tool-Integrated Reasoning
Luan Zhang, Dandan Song, Zhijing Wu +8
Tool-integrated reasoning (TIR) enables large language models (LLMs) to enhance their capabilities by interacting with external tools, such as code interpreters (CI). Most recent s…
HeavySkill: Heavy Thinking as the Inner Skill in Agentic Harness
Jianing Wang, Linsen Guo, Zhengyu Chen +8
Recent advances in agentic harness with orchestration frameworks that coordinate multiple agents with memory, skills, and tool use have achieved remarkable success in complex reaso…
When the Strongest Teacher Is Not the Best Teacher: Student-Centric Answer Selection
Zhengyu Hu, Zheyuan Xiao, Linxin Song +10
LLM training increasingly relies on teacher-generated supervision, from synthetic responses to reasoning traces and tool-use demonstrations. Current practice often chooses the high…
On the Role of Reasoning Patterns in the Generalization Discrepancy of Long Chain-of-Thought Supervised Fine-Tuning
Zhaoyi Li, Xiangyu Xi, Zhengyu Chen +6
Supervised Fine-Tuning (SFT) on long Chain-of-Thought (CoT) trajectories has become a pivotal phase in building large reasoning models. However, how CoT trajectories from different…
LongCat-Flash-Prover: Advancing Native Formal Reasoning via Agentic Tool-Integrated Reinforcement Learning
Jianing Wang, Jianfei Zhang, Qi Guo +24
We introduce LongCat-Flash-Prover, a flagship 560-billion-parameter open-source Mixture-of- Experts (MoE) model that advances Native Formal Reasoning in Lean4 through agentic tool-…
SkillCraft: Can LLM Agents Learn to Use Tools Skillfully?
Shiqi Chen, Jingze Gai, Ruochen Zhou +13
Real-world tool-using agents operate over long-horizon workflows with recurring structure and diverse demands, where effective behavior requires not only invoking atomic tools but…
TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training
Jinluan Yang, Yuxin Liu, Zhengyu Chen +7
Training tool-use agents typically relies on outcome-based filtering: Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected…
Pushing the Boundaries of Natural Reasoning: Interleaved Bonus from Formal-Logic Verification
Chuxue Cao, Jinluan Yang, Haoran Li +8
Large Language Models (LLMs) show remarkable capabilities, yet their stochastic next-token prediction creates logical inconsistencies and reward hacking that formal symbolic system…
Reinforcement Learning for Tool-Integrated Interleaved Thinking towards Cross-Domain Generalization
Zhengyu Chen, Jinluan Yang, Teng Xiao +6
Recent advances in large language models (LLMs) have demonstrated remarkable capabilities in reasoning and tool utilization. However, the generalization of tool-augmented reinforce…
TreeMind: Automatically Reproducing Android Bug Reports via LLM-empowered Monte Carlo Tree Search
Zhengyu Chen, Zhaoyi Meng, Wenxiang Zhao +5
Automatically reproducing Android app crashes from textual bug reports is challenging, particularly when the reports are incomplete and the modern UI exhibits high combinatorial co…
Sub-Scaling Laws: On the Role of Data Density and Training Strategies in LLMs
Zhengyu Chen, Siqi Wang, Teng Xiao +5
Traditional scaling laws in natural language processing suggest that increasing model size and training data enhances performance. However, recent studies reveal deviations, partic…
Does Learning Mathematical Problem-Solving Generalize to Broader Reasoning?
Ruochen Zhou, Minrui Xu, Shiqi Chen +5
There has been a growing interest in enhancing the mathematical problem-solving (MPS) capabilities of large language models. While the majority of research efforts concentrate on c…
From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling
Zhengyu Chen, Yudong Wang, Teng Xiao +5
Recent advancements in improving the reasoning capabilities of Large Language Models have underscored the efficacy of Process Reward Models (PRMs) in addressing intermediate errors…
SampleMix: A Sample-wise Pre-training Data Mixing Strategey by Coordinating Data Quality and Diversity
Xiangyu Xi, Deyang Kong, Jian Yang +7
Existing pretraining data mixing methods for large language models (LLMs) typically follow a domain-wise methodology, a top-down process that first determines domain weights and th…