7 papers · 1 filter
Do LLMs Need to See Everything? A Benchmark and Study of Failures in LLM-driven Smartphone Automation using Screentext vs. Screenshots
Shiquan Zhang, Tianyi Zhang, Le Fang +3
With the rapid advancement of large language models (LLMs), mobile agents have emerged as promising tools for phone automation, simulating human interactions on screens to accompli…
Behavioral Indicators of Loneliness: Predicting University Students' Loneliness Scores from Smartphone Sensing Data
Qianjie Wu, Tianyi Zhang, Hong Jia +1
Loneliness is a critical mental health issue among university students, yet traditional monitoring methods rely primarily on retrospective self-reports and often lack real-time beh…
AWARE Narrator and the Utilization of Large Language Models to Extract Behavioral Insights from Smartphone Sensing Data
Tianyi Zhang, Miu Kojima, Simon D'Alfonso
Smartphones, equipped with an array of sensors, have become valuable tools for personal sensing. Particularly in digital health, smartphones facilitate the tracking of health-relat…
AutoJournaling: A Context-Aware Journaling System Leveraging MLLMs on Smartphone Screenshots
Tianyi Zhang, Shiquan Zhang, Le Fang +3
Journaling offers significant benefits, including fostering self-reflection, enhancing writing skills, and aiding in mood monitoring. However, many people abandon the practice beca…
Predicting Affective States from Screen Text Sentiment
Songyan Teng, Tianyi Zhang, Simon D'Alfonso +1
The proliferation of mobile sensing technologies has enabled the study of various physiological and behavioural phenomena through unobtrusive data collection from smartphone sensor…
Enabling On-Device LLMs Personalization with Smartphone Sensing
Shiquan Zhang, Ying Ma, Le Fang +3
This demo presents a novel end-to-end framework that combines on-device large language models (LLMs) with smartphone sensing technologies to achieve context-aware and personalized…