10 papers
Role Steering of Language Models for Social Simulations
Isaac Song, Mohammed Rehan Parwani, Glenn Matlin +8
Social simulations built from language-model agents need role-conditioned behavior that can be checked before agents are placed into a simulated population. We introduce an activat…
No One Wins in Nuclear War: A Social Simulation of Military Decision-making
Glenn Matlin, Isaac Song, Anthony Wen-Ming Zang +1
WOPR is a social-simulation environment for studying how organizations make high-stakes decisions, built on a deterministic, replay-validated rules engine and using wargames as the…
Capability Provenance in Language Models: A Case Study in Social Reasoning
Glenn Matlin, Chandreyi Chakraborty, Saehee Eom +8
We use training-data attribution as an interpretable tool for capability discovery, mapping which regions of the pretraining corpus support social reasoning versus STEM reasoning i…
Trust by Design: Skill Profiles for Transparent, Cost-Aware LLM Routing
Mika Okamoto, Ansel Kaplan Erol, Glenn Matlin
How should Large Language Model (LLM) practitioners select the right model for a task without wasting money? We introduce BELLA (Budget-Efficient LLM Selection via Automated skill-…
FinForge: Semi-Synthetic Financial Benchmark Generation
Glenn Matlin, Akhil Theerthala, Anant Gupta +4
Evaluating Language Models (LMs) in specialized, high-stakes domains such as finance remains a significant challenge due to the scarcity of open, high-quality, and domain-specific…
Financial Instruction Following Evaluation (FIFE)
Glenn Matlin, Siddharth, Anirudh JM +3
Language Models (LMs) struggle with complex, interdependent instructions, particularly in high-stakes domains like finance where precision is critical. We introduce FIFE, a novel,…
Shall We Play a Game? Language Models for Open-ended Wargames
Glenn Matlin, Isaac Song, Yixiong Hao +8
LLM-based social simulations can make a generated transcript look like a single behavioral signal, but the model behind that transcript may be doing several different jobs: choosin…
Do Language Models Agree with Human Perceptions of Suspense in Stories?
Glenn Matlin, Devin Zhang, Rodrigo Barroso Loza +4
Suspense is an affective response to narrative text that is believed to involve complex cognitive processes in humans. Several psychological models have been developed to describe…
Finance Language Model Evaluation (FLaME)
Glenn Matlin, Mika Okamoto, Huzaifa Pardawala +2
Language Models (LMs) have demonstrated impressive capabilities with core Natural Language Processing (NLP) tasks. The effectiveness of LMs for highly specialized knowledge-intensi…
UnfoldML: Cost-Aware and Uncertainty-Based Dynamic 2D Prediction for Multi-Stage Classification
Yanbo Xu, Alind Khare, Glenn Matlin +4
Machine Learning (ML) research has focused on maximizing the accuracy of predictive tasks. ML models, however, are increasingly more complex, resource intensive, and costlier to de…