24 citations · 50 across the 7 of their papers we have counts for
7 papers
GUITrans2Act: Understanding User Operational Behaviors from Mobile GUI Interactions with Vision-Language Models
Yudong Zhang, Lei Hu, Daoyang Liu +4
Understanding the digital world on mobile devices is shifting from static UI perception to dynamic action comprehension. This capability enables models to convert visual state tran…
A Text-Native Interface for Generative Video Authoring
Xingyu Bruce Liu, Mira Dontcheva, Dingzeyu Li
Everyone can write their stories in freeform text format -- it's something we all learn in school. Yet storytelling via video requires one to learn specialized and complicated tool…
CoSight: Exploring Viewer Contributions to Online Video Accessibility Through Descriptive Commenting
Ruolin wang, Xingyu Liu, Biao Wang +5
The rapid growth of online video content has outpaced efforts to make visual information accessible to blind and low vision (BLV) audiences. While professional Audio Description (A…
Proactive Conversational Agents with Inner Thoughts
Xingyu Bruce Liu, Shitao Fang, Weiyan Shi +3
One of the long-standing aspirations in conversational AI is to allow them to autonomously take initiatives in conversations, i.e., being proactive. This is especially challenging…
Interacting with Thoughtful AI
Xingyu Bruce Liu, Haijun Xia, Xiang Anthony Chen
We envision the concept of Thoughtful AI, a new human-AI interaction paradigm in which the AI behaves as a continuously thinking entity. Unlike conventional AI systems that operate…
Human I/O: Towards a Unified Approach to Detecting Situational Impairments
Xingyu Bruce Liu, Jiahao Nick Li, David Kim +2
Situationally Induced Impairments and Disabilities (SIIDs) can significantly hinder user experience in contexts such as poor lighting, noise, and multi-tasking. While prior researc…
CrossA11y: Identifying Video Accessibility Issues via Cross-modal Grounding
Xingyu "Bruce" Liu, Ruolin Wang, Dingzeyu Li +2
Authors make their videos visually accessible by adding audio descriptions (AD), and auditorily accessible by adding closed captions (CC). However, creating AD and CC is challengin…