4 papers · 1 filter
From User Interface to Agent Interface: Efficiency Optimization of UI Representations for LLM Agents
Dezhi Ran, Zhi Gong, Yuzhe Guo +10
While Large Language Model (LLM) agents show great potential for automated UI navigation such as automated UI testing and AI assistants, their efficiency has been largely overlooke…
iDETEX: Empowering MLLMs for Intelligent DETailed EXplainable IQA
Zhaoran Zhao, Xinli Yue, Jianhui Sun +5
Image Quality Assessment (IQA) has progressed from scalar quality prediction to more interpretable, human-aligned evaluation paradigms. In this work, we address the emerging challe…
Instruction-augmented Multimodal Alignment for Image-Text and Element Matching
Xinli Yue, JianHui Sun, Junda Lu +6
With the rapid advancement of text-to-image (T2I) generation models, assessing the semantic alignment between generated images and text descriptions has become a significant resear…
Beyond Pass or Fail: Multi-Dimensional Benchmarking of Foundation Models for Goal-based Mobile UI Navigation
Dezhi Ran, Mengzhou Wu, Hao Yu +15
Recent advances of foundation models (FMs) have made navigating mobile applications (apps) based on high-level goal instructions within reach, with significant industrial applicati…