5 papers
Sidekick: Designing Communication for Effective Multitasking with Computer Use Agents
Ruei-Che Chang, Wenqian Xu, Dingzeyu Li +2
Computer Use Agents (CUAs) can autonomously execute complex, multi-step tasks within GUIs, enhancing efficiency through parallel multitasking. However, our formative studies with C…
StateScribe: Towards Accessible Change Awareness Across Real-World Revisits
Ruei-Che Chang, Xirui Jiang, Rosiana Natalie +6
Real-world environments evolve continuously, yet blind and low-vision (BLV) individuals often have limited access to understanding how they change over time. Unexpected or relocate…
TouchScribe: Augmenting Non-Visual Hand-Object Interactions with Automated Live Visual Descriptions
Ruei-Che Chang, Rosiana Natalie, Wenqian Xu +4
People who are blind or have low vision regularly use their hands to interact with the physical world to gain access to objects' shape, size, weight, and texture. However, many ric…
Not There Yet: Evaluating Vision Language Models in Simulating the Visual Perception of People with Low Vision
Rosiana Natalie, Wenqian Xu, Ruei-Che Chang +2
Advances in vision language models (VLMs) have enabled the simulation of general human behavior through their reasoning and problem solving capabilities. However, prior research ha…
Probing the Gaps in ChatGPT Live Video Chat for Real-World Assistance for People who are Blind or Visually Impaired
Ruei-Che Chang, Rosiana Natalie, Wenqian Xu +2
Recent advancements in large multimodal models have provided blind or visually impaired (BVI) individuals with new capabilities to interpret and engage with the real world through…