1 citations · 1 across the 1 of their papers we have counts for
2 papers
cs.CL2022★ 1 cited
MUG: Interactive Multimodal Grounding on User Interfaces
Tao Li, Gang Li, Jingjie Zheng +2
We present MUG, a novel interactive task for multimodal grounding where a user and an agent work collaboratively on an interface screen. Prior works modeled multimodal UI grounding…
cs.LG2020
Widget Captioning: Generating Natural Language Description for Mobile User Interface Elements
Yang Li, Gang Li, Luheng He +3
Natural language descriptions of user interface (UI) elements such as alternative text are crucial for accessibility and language-based interaction in general. Yet, these descripti…