1 citations · 2 across the 13 of their papers we have counts for
13 papers
TaskSense: Cognitive Chain Modeling and Difficulty Estimation for GUI Tasks
Yiwen Yin, Zhian Hu, Xiaoxi Xu +4
Measuring GUI task difficulty is crucial for user behavior analysis and agent capability evaluation. Yet, existing benchmarks typically quantify difficulty based on motor actions (…
TextOnly: A Unified Function Portal for Text-Related Functions on Smartphones
Minghao Tu, Chun Yu, Xiyuan Shen +3
Text boxes serve as portals to diverse functionalities in today's smartphone applications. However, when it comes to specific functionalities, users always need to navigate through…
MOAT: Evaluating LMMs for Capability Integration and Instruction Grounding
Zhoutong Ye, Mingze Sun, Huan-ang Gao +9
Large multimodal models (LMMs) have demonstrated significant potential as generalists in vision-language (VL) tasks. However, adoption of LMMs in real-world tasks is hindered by th…
PoseAugment: Generative Human Pose Data Augmentation with Physical Plausibility for IMU-based Motion Capture
Zhuojun Li, Chun Yu, Chen Liang +1
The data scarcity problem is a crucial factor that hampers the model performance of IMU-based human motion capture. However, effective data augmentation for IMU-based motion captur…
Say Your Reason: Extract Contextual Rules In Situ for Context-aware Service Recommendation
Yuxuan Li, Jiahui Li, Lihang Pan +2
This paper introduces SayRea, an interactive system that facilitates the extraction of contextual rules for personalized context-aware service recommendations in mobile scenarios.…
CasualGaze: Towards Modeling and Recognizing Casual Gaze Behavior for Efficient Gaze-based Object Selection
Yingtian Shi, Yukang Yan, Zisu Li +4
We present CasualGaze, a novel eye-gaze-based target selection technique to support natural and casual eye-gaze input. Unlike existing solutions that require users to keep the eye-…
SonarWatch: Field sensing technique for smartwatches based on ultrasound and motion
Yingtian Shi, Chun Yu, Xuyang Lu +3
A smartwatch worn continuously on the wrist has the potential to perceive rich interactive gestures and natural behaviors of the user. Unfortunately, the current interaction functi…
Bridging the gap between natural user expression with complex automation programming in smart homes
Yingtian Shi, Xiaoyi Liu, Chun Yu +4
A long-standing challenge in end-user programming (EUP) is to trade off between natural user expression and the complexity of programming tasks. As large language models (LLMs) are…
A Human-Computer Collaborative Tool for Training a Single Large Language Model Agent into a Network through Few Examples
Lihang Pan, Yuxuan Li, Chun Yu +1
The capabilities of a single large language model (LLM) agent for solving a complex task are limited. Connecting multiple LLM agents to a network can effectively improve overall pe…
AutoTask: Executing Arbitrary Voice Commands by Exploring and Learning from Mobile GUI
Lihang Pan, Bowen Wang, Chun Yu +3
Voice command interfaces (VCIs) have gained increasing importance, enabling hands-free and eyes-free interaction with digital devices. However, the inherent complexity in construct…
UbiPhysio: Support Daily Functioning, Fitness, and Rehabilitation with Action Understanding and Feedback in Natural Language
Chongyang Wang, Yuan Feng, Lingxiao Zhong +8
We introduce UbiPhysio, a milestone framework that delivers fine-grained action description and feedback in natural language to support people's daily functioning, fitness, and reh…
Enabling Voice-Accompanying Hand-to-Face Gesture Recognition with Cross-Device Sensing
Zisu Li, Cheng Liang, Yuntao Wang +5
Gestures performed accompanying the voice are essential for voice interaction to convey complementary semantics for interaction purposes such as wake-up state and input modality. I…
Modeling the Trade-off of Privacy Preservation and Activity Recognition on Low-Resolution Images
Yuntao Wang, Zirui Cheng, Xin Yi +7
A computer vision system using low-resolution image sensors can provide intelligent services (e.g., activity recognition) but preserve unnecessary visual privacy information from t…