51 citations · 74 across the 20 of their papers we have counts for
11 papers · 1 filter
Game-TARS: Pretrained Foundation Models for Scalable Generalist Multimodal Game Agents
Zihao Wang, Xujing Li, Yining Ye +24
We present Game-TARS, a generalist game agent trained with a unified, scalable action space anchored to human-aligned native keyboard-mouse inputs. Unlike API- or GUI-based approac…
OpenHA: A Series of Open-Source Hierarchical Agentic Models in Minecraft
Zihao Wang, Muyao Li, Kaichen He +4
The choice of action spaces is a critical yet unresolved challenge in developing capable, end-to-end trainable agents. This paper first presents a large-scale, systematic compariso…
UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning
Haoming Wang, Haoyang Zou, Huatong Song +109
The development of autonomous agents for graphical user interfaces (GUIs) presents major challenges in artificial intelligence. While recent advances in native agent models have sh…
RAM: Towards an Ever-Improving Memory System by Learning from Communications
Jiaqi Li, Xiaobo Wang, Wentao Ding +4
We introduce an innovative RAG-based framework with an ever-improving memory. Inspired by humans'pedagogical process, RAM utilizes recursively reasoning-based retrieval and experie…
Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects
Yuheng Cheng, Ceyao Zhang, Zhengwen Zhang +8
Intelligent agents stand out as a potential path toward artificial general intelligence (AGI). Thus, researchers have dedicated significant effort to diverse implementations for th…
JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models
Zihao Wang, Shaofei Cai, Anji Liu +9
Achieving human-like planning and control with multimodal observations in an open world is a key milestone for more functional generalist agents. Existing approaches can handle cer…
GROOT: Learning to Follow Instructions by Watching Gameplay Videos
Shaofei Cai, Bowei Zhang, Zihao Wang +3
We study the problem of building a controller that can follow open-ended instructions in open-world environments. We propose to follow reference videos as instructions, which offer…
MCU: An Evaluation Framework for Open-Ended Game Agents
Xinyue Zheng, Haowei Lin, Kaichen He +3
Developing AI agents capable of interacting with open-world environments to solve diverse tasks is a compelling challenge. However, evaluating such open-ended agents remains diffic…
ProAgent: Building Proactive Cooperative Agents with Large Language Models
Ceyao Zhang, Kaijie Yang, Siyi Hu +12
Building agents with adaptive behavior in cooperative tasks stands as a paramount goal in the realm of multi-agent systems. Current approaches to developing cooperative agents rely…
Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents
Zihao Wang, Shaofei Cai, Guanzhou Chen +3
We investigate the challenge of task planning for multi-task embodied agents in open-world environments. Two main difficulties are identified: 1) executing plans in an open-world e…
Open-World Multi-Task Control Through Goal-Aware Representation Learning and Adaptive Horizon Prediction
Shaofei Cai, Zihao Wang, Xiaojian Ma +2
We study the problem of learning goal-conditioned policies in Minecraft, a popular, widely accessible yet challenging open-ended environment for developing human-level multi-task a…