51 citations · 87 across the 14 of their papers we have counts for
15 papers
SurgGaze: Implicit Calibration for Accurate Gaze Analysis in Operating Rooms with Wearable Eyetrackers
Jingying Wang, Rosiana Natalie, Keyuan Hu +5
Accurate gaze tracking is essential for understanding surgeons' visual attention and cognitive processes during laparoscopic surgery, yet wearable eye trackers produce large errors…
Bespoke Visual Assistance: What and How do Blind and Low-Vision People Create with Agentic Programming?
Ellie Seehorn, Gene S-H Kim, Aziz Zeidieh +6
AI-powered assistive technologies have long supported blind and low vision (BLV) people in everyday tasks, but they are general-purpose and often fall short of meeting complex, ind…
Sidekick: Designing Communication for Effective Multitasking with Computer Use Agents
Ruei-Che Chang, Wenqian Xu, Dingzeyu Li +2
Computer Use Agents (CUAs) can autonomously execute complex, multi-step tasks within GUIs, enhancing efficiency through parallel multitasking. However, our formative studies with C…
Substantial, Decomposable, and Invisible: Visual Context Misalignment in Instructional Videos for Physical Tasks
Yayuan Li, Chenglin Li, Jingying Wang +3
Instructional videos are the dominant medium for learning physical tasks, yet they rarely match the user's real-world visual context. Motor simulation and cognitive load theories p…
Beyond Screenshots: Evaluating VLMs' Understanding of UI Animations
Chen Liang, Xirui Jiang, Naihao Deng +2
AI agents operating on user interfaces must understand how interfaces communicate state and feedback to act reliably. As a core communicative modality, animations are increasingly…
StateScribe: Towards Accessible Change Awareness Across Real-World Revisits
Ruei-Che Chang, Xirui Jiang, Rosiana Natalie +6
Real-world environments evolve continuously, yet blind and low-vision (BLV) individuals often have limited access to understanding how they change over time. Unexpected or relocate…
A11y-CUA Dataset: Characterizing the Accessibility Gap in Computer Use Agents
Ananya Gubbi Mohanbabu, Rosiana Natalie, Brandon Kim +2
Computer Use Agents (CUAs) operate interfaces by pointing, clicking, and typing -- mirroring interactions of sighted users (SUs) who can thus monitor CUAs and share control. CUAs d…
TouchScribe: Augmenting Non-Visual Hand-Object Interactions with Automated Live Visual Descriptions
Ruei-Che Chang, Rosiana Natalie, Wenqian Xu +4
People who are blind or have low vision regularly use their hands to interact with the physical world to gain access to objects' shape, size, weight, and texture. However, many ric…
Auditorily Embodied Conversational Agents: Effects of Spatialization and Situated Audio Cues on Presence and Social Perception
Yi Fei Cheng, Jarod Bloch, Alexander Wang +5
Embodiment can enhance conversational agents, such as increasing their perceived presence. This is typically achieved through visual representations of a virtual body; however, vis…
Not There Yet: Evaluating Vision Language Models in Simulating the Visual Perception of People with Low Vision
Rosiana Natalie, Wenqian Xu, Ruei-Che Chang +2
Advances in vision language models (VLMs) have enabled the simulation of general human behavior through their reasoning and problem solving capabilities. However, prior research ha…
Probing the Gaps in ChatGPT Live Video Chat for Real-World Assistance for People who are Blind or Visually Impaired
Ruei-Che Chang, Rosiana Natalie, Wenqian Xu +2
Recent advancements in large multimodal models have provided blind or visually impaired (BVI) individuals with new capabilities to interpret and engage with the real world through…
HandProxy: Expanding the Affordances of Speech Interfaces in Immersive Environments with a Virtual Proxy Hand
Chen Liang, Yuxuan Liu, Martez Mott +1
Hand interactions are increasingly used as the primary input modality in immersive environments, but they are not always feasible due to situational impairments, motor limitations,…
Audio Description Customization
Rosiana Natalie, Ruei-Che Chang, Smitha Sheshadri +2
Blind and low-vision (BLV) people use audio descriptions (ADs) to access videos. However, current ADs are unalterable by end users, thus are incapable of supporting BLV individuals…
EditScribe: Non-Visual Image Editing with Natural Language Verification Loops
Ruei-Che Chang, Yuxuan Liu, Lotus Zhang +1
Image editing is an iterative process that requires precise visual evaluation and manipulation for the output to match the editing intent. However, current image editing tools do n…
WorldScribe: Towards Context-Aware Live Visual Descriptions
Ruei-Che Chang, Yuxuan Liu, Anhong Guo
Automated live visual descriptions can aid blind people in understanding their surroundings with autonomy and independence. However, providing descriptions that are rich, contextua…