Dream-Butterfly: Configuring Embodied Conversational Guidance for Public Outdoor Mixed Reality Exhibitions

Abstract

Public outdoor mixed reality (MR) exhibitions make guidance a spatial interaction problem: visitors need timely explanations while roaming open sites, attending to virtual artworks, hazards, bystanders, and route choices. We present Dream-Butterfly, a field-deployed embodied conversational guide for campus-scale outdoor MR exhibitions. Visitors explicitly summon the lightweight non-humanoid guide; it returns visibly to a hand-relative dialogue position and answers using retrieval-augmented responses scoped by the MR runtime to the artwork currently encountered. Within the resulting mixed guidance ecology, staff remain accountable for safety, wayfinding, device support, and contingencies. Dream-Butterfly was deployed with over 30 spatially anchored artworks across a 26,000 m${}^2$ campus site, which we use as a stress case for public MR guidance under walking, field-of-view limits, route choice, bystander exposure, and safety constraints. In an in-the-wild study with 24 visitors, we compared two role arrangements: agent-default, where the embodied guide served as the default channel for artwork interpretation, and docent-default, where staff provided primary narration while the agent remained optional. Agent-default guidance was associated with more available, visitor-timed explanations, higher immersion/engagement, and stronger hedonic quality, while increasing visitors' self-curation burden around pacing, attention management, and question formulation. We argue that making interpretation available at visitors' chosen moments improves the timing of explanation in outdoor MR and shifts more work to visitors: deciding when to stop, what to ask, how deeply to engage, and when to move on.

Accepted at ACM VRST 2026. Updated to the camera-ready version. 15 pages including 4 pages of supplementary material; 6 figures