143 citations · 371 across the 21 of their papers we have counts for
22 papers
LEGOUI: Designing with UI-DSL Bricks to Balance Transparency and Controllability
Yinsi Zhou, Mingyue Yuan, Hongyue Xu +8
Generative user interface design tools enable rapid prototyping but often operate as black boxes with limited transparency and controllability. When outputs diverge from the design…
MARIA: A Framework for Marginal Risk Assessment without Ground Truth in AI Systems
Jieshan Chen, Suyu Ma, Qinghua Lu +2
Before deploying an AI system to replace an existing process, it must be compared with the incumbent to ensure improvement without added risk. Traditional evaluation relies on grou…
SHIELDA: Structured Handling of Exceptions in LLM-Driven Agentic Workflows
Jingwen Zhou, Jieshan Chen, Qinghua Lu +2
Large Language Model (LLM) agentic systems are software systems powered by LLMs that autonomously reason, plan, and execute multi-step workflows to achieve human goals, rather than…
Think Like an Engineer: A Neuro-Symbolic Collaboration Agent for Generative Software Requirements Elicitation and Self-Review
Sai Zhang, Zhenchang Xing, Jieshan Chen +5
The vision of End-User Software Engineering (EUSE) is to empower non-professional users with full control over the software development lifecycle. It aims to enable users to drive…
FactFlow: Automatic Fact Sheet Generation and Customization from Tabular Dataset via AI Chain Design & Implementation
Minh Duc Vu, Jieshan Chen, Zhenchang Xing +3
With the proliferation of data across various domains, there is a critical demand for tools that enable non-experts to derive meaningful insights without deep data analysis skills.…
From Exploration to Revelation: Detecting Dark Patterns in Mobile Apps
Jieshan Chen, Zhen Wang, Jiamou Sun +5
Mobile apps are essential in daily life but frequently employ deceptive patterns, such as visual emphasis or linguistic nudging, to manipulate user behavior. Existing research larg…
A Taxonomy of Architecture Options for Foundation Model-based Agents: Analysis and Decision Model
Jingwen Zhou, Qinghua Lu, Jieshan Chen +4
The rapid advancement of AI technology has led to widespread applications of agent systems across various domains. However, the need for detailed architecture design poses signific…
MAxPrototyper: A Multi-Agent Generation System for Interactive User Interface Prototyping
Mingyue Yuan, Jieshan Chen, Aaron Quigley
In automated user interactive design, designers face key challenges, including accurate representation of user intent, crafting high-quality components, and ensuring both aesthetic…
The Invisible Game on the Internet: A Case Study of Decoding Deceptive Patterns
Zewei Shi, Ruoxi Sun, Jieshan Chen +2
Deceptive patterns are design practices embedded in digital platforms to manipulate users, representing a widespread and long-standing issue in the web and mobile software developm…
GPTVoiceTasker: Advancing Multi-step Mobile Task Efficiency Through Dynamic Interface Exploration and Learning
Minh Duc Vu, Han Wang, Zhuang Li +4
Virtual assistants have the potential to play an important role in helping users achieves different tasks. However, these systems face challenges in their real-world usability, cha…
Designing with Language: Wireframing UI Design Intent with Generative Large Language Models
Sidong Feng, Mingyue Yuan, Jieshan Chen +2
Wireframing is a critical step in the UI design process. Mid-fidelity wireframes offer more impactful and engaging visuals compared to low-fidelity versions. However, their creatio…
Let's Discover More API Relations: A Large Language Model-based AI Chain for Unsupervised API Relation Inference
Qing Huang, Yanbang Sun, Zhenchang Xing +5
APIs have intricate relations that can be described in text and represented as knowledge graphs to aid software engineering tasks. Existing relation extraction methods have limitat…
Let's Chat to Find the APIs: Connecting Human, LLM and Knowledge Graph through AI Chain
Qing Huang, Zhenyu Wan, Zhenchang Xing +4
API recommendation methods have evolved from literal and semantic keyword matching to query expansion and query clarification. The latest query clarification method is knowledge gr…
Unveiling the Tricks: Automated Detection of Dark Patterns in Mobile Applications
Jieshan Chen, Jiamou Sun, Sidong Feng +4
Mobile apps bring us many conveniences, such as online shopping and communication, but some use malicious designs called dark patterns to trick users into doing things that are not…
Towards Real Smart Apps: Investigating Human-AI Interactions in Smartphone On-Device AI Apps
Jason Ching Yuen Siu, Jieshan Chen, Yujin Huang +2
With the emergence of deep learning techniques, smartphone apps are now embedded on-device AI features for enabling advanced tasks like speech translation, to attract users and inc…
Enhancing Virtual Assistant Intelligence: Precise Area Targeting for Instance-level User Intents beyond Metadata
Mengyu Chen, Zhenchang Xing, Jieshan Chen +2
Virtual assistants have been widely used by mobile phone users in recent years. Although their capabilities of processing user intents have been developed rapidly, virtual assistan…
Prompt Sapper: A LLM-Empowered Production Tool for Building AI Chains
Yu Cheng, Jieshan Chen, Qing Huang +3
The emergence of foundation models, such as large language models (LLMs) GPT-4 and text-to-image models DALL-E, has opened up numerous possibilities across various domains. People…
Is It a Trap? A Large-scale Empirical Study And Comprehensive Assessment of Online Automated Privacy Policy Generators for Mobile Apps
Shidong Pan, Dawen Zhang, Mark Staples +4
Privacy regulations protect and promote the privacy of individuals by requiring mobile apps to provide a privacy policy that explains what personal information is collected and how…
Extracting Replayable Interactions from Videos of Mobile App Usage
Jieshan Chen, Amanda Swearngin, Jason Wu +3
Screen recordings of mobile apps are a popular and readily available way for users to share how they interact with apps, such as in online tutorial videos, user reviews, or as atta…
Wireframe-Based UI Design Search Through Image Autoencoder
Jieshan Chen, Chunyang Chen, Zhenchang Xing +4
UI design is an integral part of software development. For many developers who do not have much UI design experience, exposing them to a large database of real-application UI desig…
Object Detection for Graphical User Interface: Old Fashioned or Deep Learning or a Combination?
Jieshan Chen, Mulong Xie, Zhenchang Xing +4
Detecting Graphical User Interface (GUI) elements in GUI images is a domain-specific object detection task. It supports many software engineering tasks, such as GUI animation and t…
Unblind Your Apps: Predicting Natural-Language Labels for Mobile GUI Components by Deep Learning
Jieshan Chen, Chunyang Chen, Zhenchang Xing +4
According to the World Health Organization(WHO), it is estimated that approximately 1.3 billion people live with some forms of vision impairment globally, of whom 36 million are bl…