Memento: Towards Proactive Visualization of Everyday Memories with Personal Wearable AR Assistant

Abstract

We introduce Memento, a conversational AR assistant that permanently captures and memorizes user's verbal queries alongside their spatiotemporal and activity contexts. By storing these "memories," Memento discovers connections between users' recurring interests and the contexts that trigger them. Upon detection of similar or identical spatiotemporal activity, Memento proactively recalls user interests and delivers up-to-date responses through AR, seamlessly integrating AR experience into their daily routine. Unlike prior work, each interaction in Memento is not a transient event, but a connected series of interactions with coherent long--term perspective, tailored to the user's broader multimodal (visual, spatial, temporal, and embodied) context. We conduct a preliminary evaluation through user feedbacks with participants of diverse expertise in immersive apps, and explore the value of proactive context-aware AR assistant in everyday settings. We share our findings and challenges in designing a proactive, context-aware AR system.

8 pages, 5 figures. This is the author's version of the article that appeared at the IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (IEEE VRW) 2026

References in corpus (11)

• GazePointAR: A Context-Aware Multimodal Voice Assistant for Pronoun Disambiguation in Wearable Augmented Reality
• Design Patterns for Situated Visualization in Augmented Reality
• Augmented Object Intelligence with XR-Objects
• Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentation
• AiGet: Transforming Everyday Moments into Hidden Knowledge Discovery with AI Assistance on Smart Glasses
• Explainable XR: Understanding User Behaviors of XR Environments using LLM-assisted Analytics Framework
• Wearable AR in Everyday Contexts: Insights from a Digital Ethnography of YouTube Videos
• Sensible Agent: A Framework for Unobtrusive Interaction with Proactive AR Agents
• MaRginalia: Enabling In-person Lecture Capturing and Note-taking Through Mixed Reality
• SituationalLLM: Proactive language models with scene awareness for dynamic, contextual task guidance
• Leveraging Vision-Language Models for Open-Vocabulary Instance Segmentation and Tracking