5 papers
From Interpretability to Control: Insights from Six Years of the TrustNLP Workshop
Rahul Gupta, Abhinav Mohanty, Anaelia Ovalle +10
The Workshop on Trustworthy Natural Language Processing (TrustNLP), co-located with major ACL conferences since 2021, has grown from 8 proceedings papers to 41 over six editions, d…
CustomerSim: Benchmarking and Aligning Multimodal Language Models as Retail User Simulators
Yada Pruksachatkun, Yixin Wan, Elaine Wan +4
We present CustomerSim, an environment and benchmark to evaluate the extent to which Multimodal Large Language Models (MLLMs) can simulate realistic, persona-driven customer behavi…
InterviewSim: A Scalable Framework for Interview-Grounded Personality Simulation
Yu Li, Pranav Narayanan Venkit, Yada Pruksachatkun +1
Simulating real personalities with large language models requires grounding generation in authentic personal data. Existing evaluation approaches rely on demographic surveys, perso…
The Need for a Socially-Grounded Persona Framework for User Simulation
Pranav Narayanan Venkit, Yu Li, Yada Pruksachatkun +1
Synthetic personas are widely used to condition large language models (LLMs) for social simulation, yet most personas are still constructed from coarse sociodemographic attributes…
Establishing Best Practices for Building Rigorous Agentic Benchmarks
Yuxuan Zhu, Tengjun Jin, Yada Pruksachatkun +22
Benchmarks are essential for quantitatively tracking progress in AI. As AI agents become increasingly capable, researchers and practitioners have introduced agentic benchmarks to e…