4 papers
Lost in the Request: How Communication Variation Disrupts Retrieval and Action in Email Agents
Feng Chen, Ritam Dutt, Atnaz Taheri +1
An email assistant should not complete less work simply because a user phrases the same request differently. Yet most benchmarks test each task with only one canonical request, lea…
PERSONAJUDGE: Simulating Individual Human Preference Judgments with Evaluator-Specific Demonstration Data
Zeyu He, Xuan Qi, Subramanian Chidambaram +4
Large language models increasingly serve as judges in AI evaluation, but current approaches rely on consensus preferences that ignore individual evaluator variation. We propose a n…
Users Mispredict Their Own Preferences for AI Writing Assistance
Vivian Lai, Zana Buçinca, Nil-Jana Akpinar +5
Proactive AI writing assistants need to predict when users want drafting help, yet we lack empirical understanding of what drives preferences. Through a factorial vignette study wi…
PILOT: Steering Synthetic Data Generation with Psychological & Linguistic Output Targeting
Caitlin Cisar, Emily Sheffield, Joshua Drake +5
Generative AI applications commonly leverage user personas as a steering mechanism for synthetic data generation, but reliance on natural language representations forces models to…