9 papers
Looks the Same, Answers Differently: Flip-Direction Steering for Robust Vision-Language Reasoning
Yeonsung Jung, Joonhyun Jeong, Hoang Pham +4
Vision-language models (VLMs) achieve strong visual reasoning performance, yet subtle changes from routine image capture and processing can alter their reasoning trajectories even…
AdvantageFlow: Regularized Advantage-Weighted RL in Flow Models
Branislav Kveton, Anup Rao, Subhojyoti Mukherjee +2
We present AdvantageFlow, a forward-process reinforcement learning (RL) algorithm for rectified flow models. The algorithm minimizes an advantage-weighted prediction loss, which ma…
Partial Policy Gradients for RL in LLMs
Puneet Mathur, Branislav Kveton, Subhojyoti Mukherjee +1
Reinforcement learning is a framework for learning to act sequentially in an unknown environment. We propose a natural approach for modeling policy structure in policy gradients. T…
No Prompt Left Behind: Exploiting Zero-Variance Prompts in LLM Reinforcement Learning via Entropy-Guided Advantage Shaping
Thanh-Long V. Le, Myeongho Jeon, Kim Vu +2
Reinforcement Learning with Verifiable Rewards (RLVR) is a powerful framework for improving the reasoning abilities of Large Language Models (LLMs). However, current methods such a…
Language Model Probabilities are Not Calibrated in Numeric Contexts
Charles Lovering, Michael Krumdick, Viet Dac Lai +5
Some statements have one well-defined continuation (e.g., "the Eiffel Tower is in [Paris]"), whereas others have a natural distribution over multiple options (e.g., "the weighted c…
An Analysis of Multilingual FActScore
Kim Trong Vu, Michael Krumdick, Varshini Reddy +2
FActScore has gained popularity as a metric to estimate the factuality of long-form texts generated by Large Language Models (LLMs) in English. However, there has not been any work…
SEC-QA: A Systematic Evaluation Corpus for Financial QA
Viet Dac Lai, Michael Krumdick, Charles Lovering +3
The financial domain frequently deals with large numbers of long documents that are essential for daily operations. Significant effort is put towards automating financial data anal…
DocFinQA: A Long-Context Financial Reasoning Dataset
Varshini Reddy, Rik Koncel-Kedziorski, Viet Dac Lai +3
For large language models (LLMs) to be effective in the financial domain -- where each decision can have a significant impact -- it is necessary to investigate realistic tasks and…
BizBench: A Quantitative Reasoning Benchmark for Business and Finance
Rik Koncel-Kedziorski, Michael Krumdick, Viet Lai +3
Answering questions within business and finance requires reasoning, precision, and a wide-breadth of technical knowledge. Together, these requirements make this domain difficult fo…