Learning GUI Grounding with Spatial Reasoning from Visual Feedback

Yu Zhao, Wei-Ning Chen, Huseyin Atahan Inan +8

Graphical User Interface (GUI) grounding is commonly framed as a coordinate prediction task -- given a natural language instruction, generate on-screen coordinates for actions such…