Large Language Models (LLMs) as Agents for Augmented Democracy

Abstract

We explore an augmented democracy system built on off-the-shelf LLMs fine-tuned to augment data on citizen's preferences elicited over policies extracted from the government programs of the two main candidates of Brazil's 2022 presidential election. We use a train-test cross-validation setup to estimate the accuracy with which the LLMs predict both: a subject's individual political choices and the aggregate preferences of the full sample of participants. At the individual level, we find that LLMs predict out of sample preferences more accurately than a "bundle rule", which would assume that citizens always vote for the proposals of the candidate aligned with their self-reported political orientation. At the population level, we show that a probabilistic sample augmented by an LLM provides a more accurate estimate of the aggregate preferences of a population than the non-augmented probabilistic sample alone. Together, these results indicates that policy preference data augmented using LLMs can capture nuances that transcend party lines and represents a promising avenue of research for data augmentation.

24 pages main manuscript with 4 figures. 13 pages of supplementary material

References in corpus (11)

• Llama 2: Open Foundation and Fine-Tuned Chat Models
• Gender bias and stereotypes in Large Language Models
• Mistral 7B
• AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation
• The Falcon Series of Open Language Models
• Large Language Model based Multi-Agents: A Survey of Progress and Challenges
• ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate
• Systematic Biases in LLM Simulations of Debates
• LLM Voting: Human Choices and AI Collective Decision Making
• Simulating Human Strategic Behavior: Comparing Single and Multi-agent LLMs
• Humans or LLMs as the Judge? A Study on Judgement Biases