8 papers
MCP-Universe RL: A Framework for Training MCP Tool-Use Agents via Reinforcement Learning
Ziyang Luo, Yan Yang, Xiangru Jian +5
Reinforcement learning (RL) has become an effective way to improve the tool-use ability of large language models (LLMs), but most existing RL frameworks stop at the policy update.…
StateAct: Program State, before Pixels, for Long-Horizon Computer-Use Agents
Yan Yang, Xiangru Jian, Ziyang Luo +7
Computer-use agents are usually improved by strengthening perception: better models for reading a screenshot and choosing where to click. Yet a screenshot is only a lossy rendering…
Grounding Computer Use Agents on Human Demonstrations
Aarash Feizi, Shravan Nayak, Xiangru Jian +14
Building reliable computer-use agents requires grounding: accurately connecting natural language instructions to the correct on-screen elements. While large datasets exist for web…
CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use Agents
Xiangru Jian, Shravan Nayak, Kevin Qinghong Lin +5
Computer-use agents (CUAs) hold great promise for automating complex desktop workflows, yet progress toward general-purpose agents is bottlenecked by the scarcity of continuous, hi…
AlignVLM: Bridging Vision and Language Latent Spaces for Multimodal Document Understanding
Ahmed Masry, Juan A. Rodriguez, Tianyu Zhang +19
Aligning visual features with language embeddings is a key challenge in vision-language models (VLMs). The performance of such models hinges on having a good connector that maps vi…
Scope: Selective Cross-modal Orchestration of Visual Perception Experts
Tianyu Zhang, Suyuchen Wang, Chao Wang +5
Vision-language models (VLMs) benefit from multiple vision encoders, but naively stacking them yields diminishing returns while multiplying inference costs. We propose SCOPE, a Mix…