From the 1 of 10 linked papers with an AI index.
4 papers · 1 filter
PiCSAR: Probabilistic Confidence Selection And Ranking for Reasoning Chains
Joshua Ong Jun Leang, Zheng Zhao, Aryo Pradipta Gema +7
The paper proposes PiCSAR, a training-free scoring method that uses the joint log-likelihood of reasoning steps and final answer to select the most reliable reasoning chain from mu…
OpenSIR: Open-Ended Self-Improving Reasoner
Wai-Chung Kwan, Joshua Ong Jun Leang, Pavlos Vougiouklis +3
Recent advances in large language model (LLM) reasoning through reinforcement learning rely on annotated datasets for verifiable rewards, which may limit models' ability to surpass…
SCOPE: Self-Play via Co-Evolving Policies for Open-Ended Tasks
Wai-Chung Kwan, Aryo Pradipta Gema, Joshua Ong Jun Leang +1
Self-play can train language models without external supervision. However, existing methods require rule-checkable answers, leaving open-ended tasks dependent on curated prompts or…
Are We Done with MMLU?
Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong +13
Maybe not. We identify and analyse errors in the popular Massive Multitask Language Understanding (MMLU) benchmark. Even though MMLU is widely adopted, our analysis demonstrates nu…