6 papers · 1 filter
Anamnesis: An Open-Source Platform for Large-Scale Backstory-Conditioned Survey Simulation
Song-Ze Yu, Joseph Suh, Serina Chang +1
We present Anamnesis, an interactive system for demographically controllable survey simulation using large language models. Open-source, and designed for non-technical users/resear…
Quantifying the Utility of User Simulators for Building Collaborative LLM Assistants
Joseph Suh, Ayush Raj, Minwoo Kang +1
User simulators are increasingly leveraged to build interactive AI assistants, yet how to measure the quality of these simulators remains an open question. In this work, we show ho…
Graph-Based Alternatives to LLMs for Human Simulation
Joseph Suh, Suhong Moon, Serina Chang
Large language models (LLMs) have become a popular approach for simulating human behaviors, yet it remains unclear if LLMs are necessary for all simulation tasks. We study a broad…
Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinions
Joseph Suh, Erfan Jahanparast, Suhong Moon +2
Large language models (LLMs) present novel opportunities in public opinion research by predicting survey responses in advance during the early stages of survey design. Prior method…
Valid Survey Simulations with Limited Human Data: The Roles of Prompting, Fine-Tuning, and Rectification
Stefan Krsteski, Giuseppe Russo, Serina Chang +2
Surveys provide valuable insights into public opinion and behavior, but their execution is costly and slow. Large language models (LLMs) have been proposed as a scalable, low-cost…
ChatBench: From Static Benchmarks to Human-AI Evaluation
Serina Chang, Ashton Anderson, Jake M. Hofman
With the rapid adoption of LLM-based chatbots, there is a pressing need to evaluate what humans and LLMs can achieve together. However, standard benchmarks, such as MMLU, measure L…