53 citations · 250 across the 25 of their papers we have counts for
4 papers · 1 filter
UIBert: Learning Generic Multimodal Representations for UI Understanding
Chongyang Bai, Xiaoxue Zang, Ying Xu +4
To improve the accessibility of smart devices and to simplify their usage, building models which understand user interfaces (UIs) and assist users to complete their tasks is critic…
Multimodal Icon Annotation For Mobile Applications
Xiaoxue Zang, Ying Xu, Jindong Chen
Annotating user interfaces (UIs) that involves localization and classification of meaningful UI elements on a screen is a critical step for many mobile applications such as screen…
PhotoChat: A Human-Human Dialogue Dataset with Photo Sharing Behavior for Joint Image-Text Modeling
Xiaoxue Zang, Lijuan Liu, Maria Wang +3
We present a new human-human dialogue dataset - PhotoChat, the first dataset that casts light on the photo sharing behavior in onlin emessaging. PhotoChat contains 12k dialogues, e…
ActionBert: Leveraging User Actions for Semantic Understanding of User Interfaces
Zecheng He, Srinivas Sunkara, Xiaoxue Zang +7
As mobile devices are becoming ubiquitous, regularly interacting with a variety of user interfaces (UIs) is a common aspect of daily life for many people. To improve the accessibil…