6 papers · 1 filter
Cognitive Demand Steering for Adaptive Meta-Reasoning in Large Language Models
John Scoville, Shengzhuang Chen, Yejin Bang +2
Recent meta-reasoning frameworks improve LLM reasoning by wrapping chain-of-thought generation in an iterative control loop, allowing more effective backtracking, termination of re…
Aligning Language Model Benchmarks with Pairwise Preferences
Marco Gutierrez, Xinyi Leng, Hannah Cyberey +3
Language model benchmarks are pervasive and computationally-efficient proxies for real-world performance. However, many recent works find that benchmarks often fail to predict real…
Scales++: Compute Efficient Evaluation Subset Selection with Cognitive Scales Embeddings
Andrew M. Bean, Nabeel Seedat, Shengzhuang Chen +1
The prohibitive cost of evaluating large language models (LLMs) on comprehensive benchmarks necessitates the creation of small yet representative data subsets (i.e., tiny benchmark…
To Whom Do Language Models Align? Measuring Principal Hierarchies Under High-Stakes Competing Demands
Fangyi Yu, Nabeel Seedat, Jonathan Richard Schwarz +1
Language models deployed in high-stakes professional settings face conflicting demands from users, institutional authorities, and professional norms. How models act when these dema…
Bad Students Make Great Teachers: Active Learning Accelerates Large-Scale Visual Understanding
Talfan Evans, Shreya Pathak, Hamza Merzic +3
Power-law scaling indicates that large-scale training with uniform sampling is prohibitively slow. Active learning methods aim to increase data efficiency by prioritizing learning…
Empowering Biomedical Discovery with AI Agents
Shanghua Gao, Ada Fang, Yepeng Huang +6
We envision "AI scientists" as systems capable of skeptical learning and reasoning that empower biomedical research through collaborative agents that integrate AI models and biomed…