6 papers
League: Leaderboard Generation on Demand
Jian Wu, Jiayu Zhang, Dongyuan Li +5
This paper introduces Leaderboard Auto Generation (LAG), a novel and well-organized framework for automatic generation of leaderboards on a given research topic in rapidly evolving…
How Far Are AI Scientists from Changing the World?
Qiujie Xie, Yixuan Weng, Minjun Zhu +9
The emergence of large language models (LLMs) is propelling automated scientific discovery to the next level, with LLM-based Artificial Intelligence (AI) Scientist systems now taki…
Constrain Alignment with Sparse Autoencoders
Qingyu Yin, Chak Tou Leong, Minjun Zhu +7
The alignment of large language models (LLMs) with human preferences remains a key challenge. While post-training techniques like Reinforcement Learning from Human Feedback (RLHF)…
AI Scientists Fail Without Strong Implementation Capability
Minjun Zhu, Qiujie Xie, Yixuan Weng +4
The emergence of Artificial Intelligence (AI) Scientist represents a paradigm shift in scientific discovery, with large language models (LLMs) taking the lead as the primary execut…
ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learning
Ziyu Wan, Yunxiang Li, Xiaoyu Wen +8
Recent research on Reasoning of Large Language Models (LLMs) has sought to further enhance their performance by integrating meta-thinking -- enabling models to monitor, evaluate, a…
ThinkBench: Dynamic Out-of-Distribution Evaluation for Robust LLM Reasoning
Shulin Huang, Linyi Yang, Yan Song +9
Evaluating large language models (LLMs) poses significant challenges, particularly due to issues of data contamination and the leakage of correct answers. To address these challeng…