11 papers
Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation
Chenyang Yang, Xinran Zhao, Tongshuang Wu +1
Frontier LLM agents are automating many business tasks, but their high inference cost makes large-scale deployment unsustainable. Small language models (SLMs) offer a cheaper alter…
"GenAI Defaults to Bias!" Gamify AI Literacy Through Reflections on Prompts
Qianou Ma, Megan Chai, Yike Tan +5
As Generative AI (GenAI) becomes widespread, it is increasingly important for the public to understand the model's behaviors and biases. However, existing AI literacy efforts miss…
OdysSim: Building Foundation Models for Human Behavior Simulation
Xuhui Zhou, Weiwei Sun, Weihua Du +6
Large language models are increasingly deployed as human simulators for interactive evaluation and social simulation. Yet helpfulness-driven post-training pulls them toward a homog…
Comparing Developer and LLM Biases in Code Evaluation
Aditya Mittal, Ryan Shar, Zichu Wu +6
As LLMs are increasingly used as judges in code applications, they should be evaluated in realistic interactive settings that capture partial context and ambiguous intent. We prese…
RECAP: An End-to-End Platform for Capturing, Replaying, and Analyzing AI-Assisted Programming Interactions
Keyu He, Qianou Ma, Valerie Chen +2
Understanding how developers interact with AI coding assistants requires more than chat logs or git histories in isolation; it requires reconstructing the full context: which promp…
What Prompts Don't Say: Understanding and Managing Underspecification in LLM Prompts
Chenyang Yang, Yike Shi, Qianou Ma +3
Prompt underspecification is a common challenge when interacting with LLMs. In this paper, we present an in-depth analysis of this problem, showing that while LLMs can often infer…