Ontologies in Design: How Imagining a Tree Reveals Possibilites and Assumptions in Large Language Models

Abstract

Amid the recent uptake of Generative AI, sociotechnical scholars and critics have traced a multitude of resulting harms, with analyses largely focused on values and axiology (e.g., bias). While value-based analyses are crucial, we argue that ontologies -- concerning what we allow ourselves to think or talk about -- is a vital but under-recognized dimension in analyzing these systems. Proposing a need for a practice-based engagement with ontologies, we offer four orientations for considering ontologies in design: pluralism, groundedness, liveliness, and enactment. We share examples of potentialities that are opened up through these orientations across the entire LLM development pipeline by conducting two ontological analyses: examining the responses of four LLM-based chatbots in a prompting exercise, and analyzing the architecture of an LLM-based agent simulation. We conclude by sharing opportunities and limitations of working with ontologies in the design and development of sociotechnical systems.

20 pages, 1 figure, 2 tables, CHI '25

References in corpus (8)

• Cultural Bias and Cultural Alignment of Large Language Models
• Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale
• Jury Learning: Integrating Dissenting Voices into Machine Learning Models
• Collective Constitutional AI: Aligning a Language Model with Public Input
• A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration
• Understanding the Role of Temperature in Diverse Question Generation by GPT-4
• The Power of Absence: Thinking with Archival Theory in Algorithmic Design
• The Empty Signifier Problem: Towards Clearer Paradigms for Operationalising "Alignment" in Large Language Models