Plurals: A System for Guiding LLMs Via Simulated Social Ensembles
arXiv:2409.17213 · doi:10.1145/3706598.3713675
Abstract
Recent debates raised concerns that language models may favor certain viewpoints. But what if the solution is not to aim for a 'view from nowhere' but rather to leverage different viewpoints? We introduce Plurals, a system and Python library for pluralistic AI deliberation. Plurals consists of Agents (LLMs, optionally with personas) which deliberate within customizable Structures, with Moderators overseeing deliberation. Plurals is a generator of simulated social ensembles. Plurals integrates with government datasets to create nationally representative personas, includes deliberation templates inspired by deliberative democracy, and allows users to customize both information-sharing structures and deliberation behavior within Structures. Six case studies demonstrate fidelity to theoretical constructs and efficacy. Three randomized experiments show simulated focus groups produced output resonant with an online sample of the relevant audiences (chosen over zero-shot generation in 75% of trials). Plurals is both a paradigm and a concrete system for pluralistic AI. The Plurals library is available at https://github.com/josh-ashkinaze/plurals and will be continually updated.
CHI 2025
References in corpus (29)
- Towards A Rigorous Science of Interpretable Machine Learning
- Out of One, Many: Using Language Models to Simulate Human Samples
- Constitutional AI: Harmlessness from AI Feedback
- Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing
- Towards Understanding Sycophancy in Language Models
- Jury Learning: Integrating Dissenting Voices into Machine Learning Models
- Comparing the Perceived Legitimacy of Content Moderation Processes: Contractors, Algorithms, Expert Panels, and Digital Juries
- Modular Politics: Toward a Governance Layer for Online Communities
- PolicyKit: Building Governance in Online Communities
- ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate
- DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
- PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits
- Does Writing with Language Models Reduce Content Diversity?
- Mind Your POV: Convergence of Articles and Editors Towards Wikipedia's Neutrality Norm
- A Roadmap to Pluralistic Alignment
- Debating with More Persuasive LLMs Leads to More Truthful Answers
- Cura: Curation at Social Media Scale
- When Large Language Models contradict humans? Large Language Models' Sycophantic Behaviour
- Chain of Agents: Large Language Models Collaborating on Long-Context Tasks
- Stick to your Role! Stability of Personal Values Expressed in Large Language Models
- Can LLMs Produce Faithful Explanations For Fact-checking? Towards Faithful Explainable Fact-Checking via Multi-Agent Debate
- Adapting Large Language Models for Content Moderation: Pitfalls in Data Engineering and Supervised Fine-tuning
- Case Repositories: Towards Case-Based Reasoning for AI Alignment
- Market Concentration Implications of Foundation Models
- AgentsCoMerge: Large Language Model Empowered Collaborative Decision Making for Ramp Merging
- Self-Alignment of Large Language Models via Monopolylogue-based Social Scene Simulation
- I Am Not Them: Fluid Identities and Persistent Out-group Bias in Large Language Models
- Know Your Limits: A Survey of Abstention in Large Language Models
- Coalitions of Large Language Models Increase the Robustness of AI Agents