7 papers · 1 filter
Thinking Hard, Not Smart: Reasoning Models Fail to Ration Test-Time Compute Across Questions
Chenrui Fan, Yize Cheng, Ming Li +3
Reasoning language models increasingly use test-time compute to improve performance, but existing evaluations typically study this compute one question at a time. Yet when multiple…
How Can Rhetoric Reward-Hack AI Reviewers? Dissecting Rhetorical Sensitivity in AI-Based Peer Review
Ming Li, Chenguang Wang, Xirui Li +5
As large language models increasingly participate in scientific evaluation, we investigate a potential form of reward hacking: how rhetorical choices shape AI-review judgments when…
Representing Visual Evidence for Item Difficulty Prediction: Visual Textualization and Image-Native Modeling
Han Chen, Ming Li, Hong Jiao +1
Predicting item difficulty from content can provide an initial estimate for newly developed questions before sufficient student responses are available. Existing approaches typical…
Can LLMs Really Understand Item Difficulty Levels? Implications for Automated Item Generation Using LLMs
Xinyi Wang, Hong Jiao, Ming Li +4
The estimation of item difficulty plays a key role in both formative assessment and large-scale high-stakes summative assessments. This study explores how large language models (LL…
When is Your LLM Steerable?
Chenrui Fan, Yize Cheng, Ming Li +2
Activation steering offers a lightweight approach to control language models' behavior at inference time, but whether it succeeds or fails heavily depends on the prompt, concept, m…
Automated Alignment of Math Items to Content Standards in Large-Scale Assessments Using Language Models
Qingshu Xu, Hong Jiao, Tianyi Zhou +4
Accurate alignment of items to content standards is critical for valid score interpretation in large-scale assessments. This study evaluates three automated paradigms for aligning…