Zoom to Essence: Trainless GUI Grounding by Inferring upon Interface Elements

Ziwei Liu, Tao Feng, Borui Kang +2

Multimodal Large Language Model (MLLM)-based Graphical User Interface (GUI) agents develop rapidly, with visual grounding that maps natural language instructions to target UI eleme…