5 papers
PlayWrite: A Multimodal System for AI Supported Narrative Co-Authoring Through Play in XR
Esen K. Tütüncü, Qian Zhou, Frederik Brudy +2
Current AI writing tools, which rely on text prompts, poorly support the spatial and interactive nature of storytelling where ideas emerge from direct manipulation and play. We pre…
GroundLink: Exploring How Contextual Meeting Snippets Can Close Common Ground Gaps in Editing 3D Scenes for Virtual Production
Gun Woo, Park, Frederik Brudy +2
Virtual Production (VP) professionals often face challenges accessing tacit knowledge and creative intent, which are important in forming common ground with collaborators and in co…
PrevizWhiz: Combining Rough 3D Scenes and 2D Video to Guide Generative Video Previsualization
Erzhen Hu, Frederik Brudy, David Ledo +2
In pre-production, filmmakers and 3D animation experts must rapidly prototype ideas to explore a film's possibilities before fullscale production, yet conventional approaches invol…
Augmenting speech transcripts of VR recordings with gaze, pointing, and visual context for multimodal coreference resolution
Riccardo Bovo, Frederik Brudy, George Fitzmaurice +1
Understanding transcripts of immersive multimodal conversations is challenging because speakers frequently rely on visual context and non-verbal cues, such as gestures and visual a…
IndoorWorld: Integrating Physical Task Solving and Social Simulation in A Heterogeneous Multi-Agent Environment
Dekun Wu, Frederik Brudy, Bang Liu +1
Virtual environments are essential to AI agent research. Existing environments for LLM agent research typically focus on either physical task solving or social simulation, with the…