4 papers
Generative Muscle Stimulation: Providing Users with Physical Assistance by Constraining Multimodal-AI with Embodied Knowledge
Yun Ho, Romain Nith, Peili Jiang +5
Electrical muscle stimulation (EMS) can support physical-assistance (e.g., shaking a spray-can before painting). However, EMS-assistance is highly-specialized because it is (1) fix…
Orca: Browsing at Scale Through User-Driven and AI-Facilitated Orchestration Across Malleable Webpages
Peiling Jiang, Haijun Xia
Web-based activities span multiple webpages. However, conventional browsers with stacks of tabs cannot support operating and synthesizing large volumes of information across pages.…
Gradual Generation of User Interfaces as a Design Method for Malleable Software
Bryan Min, Peiling Jiang, Zhicheng Huang +1
AI is growing increasingly capable of automatically generating user interfaces (GenUI) from user prompts. However, designing GenUI applications that enable users to discover divers…
Generative and Malleable User Interfaces with Generative and Evolving Task-Driven Data Model
Yining Cao, Peiling Jiang, Haijun Xia
Unlike static and rigid user interfaces, generative and malleable user interfaces offer the potential to respond to diverse users' goals and tasks. However, current approaches prim…