5 papers
Learning GUI Grounding with Spatial Reasoning from Visual Feedback
Yu Zhao, Wei-Ning Chen, Huseyin Atahan Inan +8
Graphical User Interface (GUI) grounding is commonly framed as a coordinate prediction task -- given a natural language instruction, generate on-screen coordinates for actions such…
Computer-Using World Model
Yiming Guan, Rui Yu, John Zhang +15
Agents operating in complex software environments benefit from reasoning about the consequences of their actions, as even a single incorrect user interface (UI) operation can derai…
Towards Active Synthetic Data Generation for Finetuning Language Models
Samuel Kessler, Menglin Xia, Daniel Madrigal Diaz +5
A common and effective means for improving language model capabilities involves finetuning a ``student'' language model's parameters on generations from a more proficient ``teacher…
Budget-Aware Agentic Routing via Boundary-Guided Training
Caiqi Zhang, Menglin Xia, Xuchao Zhang +5
As large language models (LLMs) evolve into autonomous agents that execute long-horizon workflows, invoking a high-capability model at every step becomes economically unsustainable…
Enhancing Reasoning Capabilities of Small Language Models with Blueprints and Prompt Template Search
Dongge Han, Menglin Xia, Daniel Madrigal Diaz +7
Small language models (SLMs) offer promising and efficient alternatives to large language models (LLMs). However, SLMs' limited capacity restricts their reasoning capabilities and…