31 citations · 126 across the 58 of their papers we have counts for
63 papers
PAPER2LLM++: Continual Self-Evolution of LLMs from Research Papers
Hongji Pu, Yilun Zhao, Wenpeng Yin
Research on LLMs continually uncovers model limitations, their causes, and potential solutions. Yet these human discoveries remain largely disconnected from model evolution: an LLM…
Small Language Models as Judges for Rubric-Based Reinforcement Learning
Fengyu Xie, Yilun Zhao, Bingsen Chen +2
Rubric-based reinforcement learning extends RL beyond tasks with exact answers or rule-based verifiers by scoring responses against instance-specific criteria. However, this makes…
DesignAgent3D: Interactive 3D Scene Editing via Designer-like Multimodal Reasoning
Xiujin Liu, Tianyu Yang, Yilun Zhao +1
Text guided 3D scene editing provides an intuitive interface for modifying reconstructed environments, but remains difficult because natural language design requests are often sema…
Measuring the Gap Between Human and LLM Research Ideas
Ziyu Chen, Yilun Zhao, Arman Cohan
LLMs are increasingly used to brainstorm research ideas, but existing evaluations mostly judge individual ideas by novelty, feasibility, or expert preference. We instead ask: how f…
A Survey of Multimodal Mathematical Reasoning: From Perception, Alignment to Reasoning
Tianyu Yang, Sihong Wu, Yilun Zhao +6
Multimodal Mathematical Reasoning (MMR) has recently attracted increasing attention for its capability to solve mathematical problems involving both textual and visual modalities.…
SAGE: Benchmarking and Improving Retrieval for Deep Research Agents
Tiansheng Hu, Yilun Zhao, Canyu Zhang +2
Deep research agents have emerged as powerful systems for addressing complex queries. Meanwhile, LLM-based retrievers have demonstrated strong capability in following instructions…
ANCHOR: Branch-Point Data Generation for GUI Agents
Jinbiao Wei, Yilun Zhao, Kangqi Ni +1
End-to-end GUI agents for real desktop environments require large amounts of high-quality interaction data, yet collecting human demonstrations is expensive and existing synthetic…
SciRAG: Adaptive, Citation-Aware, and Outline-Guided Retrieval and Synthesis for Scientific Literature
Hang Ding, Yilun Zhao, Tiansheng Hu +2
The accelerating growth of scientific publications has intensified the need for scalable, trustworthy systems to synthesize knowledge across diverse literature. While recent retrie…
Measuring what Matters: Construct Validity in Large Language Model Benchmarks
Andrew M. Bean, Ryan Othniel Kearns, Angelika Romanou +39
Evaluating large language models (LLMs) is crucial for both assessing their capabilities and identifying safety or robustness issues prior to deployment. Reliably measuring abstrac…
AlphaResearch: Accelerating New Algorithm Discovery with Language Models
Zhaojian Yu, Kaiyue Feng, Yilun Zhao +3
LLMs have made significant progress in complex but easy-to-verify problems, yet they still struggle with discovering the unknown. In this paper, we present \textbf{AlphaResearch},…
FinTrust: A Comprehensive Benchmark of Trustworthiness Evaluation in Finance Domain
Tiansheng Hu, Tongyan Hu, Liuyang Bai +3
Recent LLMs have demonstrated promising ability in solving finance related problems. However, applying LLMs in real-world finance application remains challenging due to its high ri…
PuzzlePlex: Benchmarking Foundation Models on Reasoning and Planning with Puzzles
Yitao Long, Yuru Jiang, Hongjun Liu +6
This work investigates the reasoning and planning capabilities of foundation models and their scalability in complex, dynamic environments. We introduce PuzzlePlex, a benchmark des…
FinLFQA: Evaluating Attributed Text Generation of LLMs in Financial Long-Form Question Answering
Yitao Long, Tiansheng Hu, Yilun Zhao +2
Large Language Models (LLMs) frequently hallucinate to long-form questions, producing plausible yet factually incorrect answers. A common mitigation strategy is to provide attribut…
MSRS: Evaluating Multi-Source Retrieval-Augmented Generation
Rohan Phanse, Yijie Zhou, Kejian Shi +4
Retrieval-augmented systems are typically evaluated in settings where information required to answer the query can be found within a single source or the answer is short-form or fa…
AbGen: Evaluating Large Language Models in Ablation Study Design and Evaluation for Scientific Research
Yilun Zhao, Weiyuan Chen, Zhijian Xu +5
We introduce AbGen, the first benchmark designed to evaluate the capabilities of LLMs in designing ablation studies for scientific research. AbGen consists of 1,500 expert-annotate…
Can Multimodal Foundation Models Understand Schematic Diagrams? An Empirical Study on Information-Seeking QA over Scientific Papers
Yilun Zhao, Chengye Wang, Chuhan Li +1
This paper introduces MISS-QA, the first benchmark specifically designed to evaluate the ability of models to interpret schematic diagrams within scientific literature. MISS-QA com…
Can LLMs Identify Critical Limitations within Scientific Research? A Systematic Evaluation on AI Research Papers
Zhijian Xu, Yilun Zhao, Manasi Patwardhan +2
Peer review is fundamental to scientific research, but the growing volume of publications has intensified the challenges of this expertise-intensive process. While LLMs show promis…
SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Tasks
Yilun Zhao, Kaiyan Zhang, Tiansheng Hu +15
We present SciArena, an open and collaborative platform for evaluating foundation models on scientific literature-grounded tasks. Unlike traditional benchmarks for scientific liter…
SUCEA: Reasoning-Intensive Retrieval for Adversarial Fact-checking through Claim Decomposition and Editing
Hongjun Liu, Yilun Zhao, Arman Cohan +1
Automatic fact-checking has recently received more attention as a means of combating misinformation. Despite significant advancements, fact-checking systems based on retrieval-augm…
Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective
Siyue Zhang, Yilun Zhao, Liyuan Geng +3
Large language model (LLM)-based embedding models, benefiting from large scale pre-training and post-training, have begun to surpass BERT and T5-based models on general-purpose tex…
Table-R1: Inference-Time Scaling for Table Reasoning
Zheyuan Yang, Lyuhao Chen, Arman Cohan +1
In this work, we present the first study to explore inference-time scaling on table reasoning tasks. We develop and evaluate two post-training strategies to enable inference-time s…
Judging with Many Minds: Do More Perspectives Mean Less Prejudice? On Bias Amplifications and Resistance in Multi-Agent Based LLM-as-Judge
Chiyu Ma, Enpei Zhang, Yilun Zhao +7
LLM-as-Judge has emerged as a scalable alternative to human evaluation, enabling large language models (LLMs) to provide reward signals in trainings. While recent work has explored…
Z1: Efficient Test-time Scaling with Code
Zhaojian Yu, Yinghao Wu, Yilun Zhao +2
Large Language Models (LLMs) can achieve enhanced complex problem-solving through test-time computing scaling, yet this often entails longer contexts and numerous reasoning token c…
Physics: Benchmarking Foundation Models on University-Level Physics Problem Solving
Kaiyue Feng, Yilun Zhao, Yixin Liu +4
We introduce Physics, a benchmark for evaluating foundation models on advanced physics problem solving. It contains 1034 annotated, open-ended problems from PhD-qualifying exams ac…