72 citations · 77 across the 4 of their papers we have counts for
6 papers
Screen2Vec: Semantic Embedding of GUI Screens and GUI Components
Toby Jia-Jun Li, Lindsay Popowski, Tom M. Mitchell +1
Representing the semantics of GUI screens and components is crucial to data-driven computational methods for modeling user-GUI interactions and mining GUI designs. Existing GUI sem…
Geno: A Developer Tool for Authoring Multimodal Interaction on Existing Web Applications
Ritam Jyoti Sarmah, Yunpeng Ding, Di Wang +3
Supporting voice commands in applications presents significant benefits to users. However, adding such support to existing GUI-based web apps is effort-consuming with a high learni…
Privacy-Preserving Script Sharing in GUI-based Programming-by-Demonstration Systems
Toby Jia-Jun Li, Jingya Chen, Brandon Canfield +1
An important concern in end user development (EUD) is accidentally embedding personal information in program artifacts when sharing them. This issue is particularly important in GU…
Towards Effective Human-AI Collaboration in GUI-Based Interactive Task Learning Agents
Toby Jia-Jun Li, Jingya Chen, Tom M. Mitchell +1
We argue that a key challenge in enabling usable and useful interactive task learning for intelligent agents is to facilitate effective Human-AI collaboration. We reflect on our pa…
Interactive Task and Concept Learning from Natural Language Instructions and GUI Demonstrations
Toby Jia-Jun Li, Marissa Radensky, Justin Jia +3
Natural language programming is a promising approach to enable end users to instruct new tasks for intelligent agents. However, our formative study found that end users would often…
Not at Home on the Range: Peer Production and the Urban/Rural Divide
Isaac Johnson, Allen Yilun Lin, Toby Jia-Jun Li +4
Wikipedia articles about places, OpenStreetMap features, and other forms of peer-produced content have become critical sources of geographic knowledge for humans and intelligent te…