8 papers
Cross-Domain Hybrid OPD for Generalizable Search Agents
Hongzhan Chen, Xiaoyu Liu, Dengming Zhang +11
Recent advances in Reinforcement Learning (RL) have substantially improved the capabilities of autonomous search agents, enabling sophisticated planning, and iterative retrieval ov…
LaoBench: A Large-Scale Multidimensional Lao Benchmark for Large Language Models
Jian Gao, Richeng Xuan, Zhaolu Kang +9
The rapid advancement of large language models (LLMs) has not been matched by their evaluation in low-resource languages, especially Southeast Asian languages like Lao. To fill thi…
FlagEval Findings Report: A Preliminary Evaluation of Large Reasoning Models on Automatically Verifiable Textual and Visual Questions
Bowen Qin, Chen Yue, Fang Yin +26
We conduct a moderate-scale contamination-free (to some extent) evaluation of current large reasoning models (LRMs) with some preliminary findings. We also release ROME, our evalua…
Beyond Multiple Choice: Verifiable OpenQA for Robust Vision-Language RFT
Yesheng Liu, Hao Li, Haiyu Xu +9
Multiple-choice question answering (MCQA) has been a popular format for evaluating and reinforcement fine-tuning (RFT) of modern multimodal language models. Its constrained output…
Beyond Solving Math Quiz: Evaluating the Ability of Large Reasoning Models to Ask for Information
Youcheng Huang, Bowen Qin, Chen Huang +3
Large Reasoning Models (LRMs) have demonstrated remarkable problem-solving abilities in mathematics, as evaluated by existing benchmarks exclusively on well-defined problems. Howev…
FlagEvalMM: A Flexible Framework for Comprehensive Multimodal Model Evaluation
Zheqi He, Yesheng Liu, Jing-shu Zheng +5
We present FlagEvalMM, an open-source evaluation framework designed to comprehensively assess multimodal models across a diverse range of vision-language understanding and generati…