17 citations · 55 across the 28 of their papers we have counts for
29 papers
Can Large Language Models Generate Observability-Aware Code?
Yongliang Tao, Hongyu Zhang, Pengfei Gao +8
Recent advances in coding agents have enabled the generation of increasingly complex software systems. While existing evaluations primarily focus on functional correctness, product…
Computer-Using World Model
Yiming Guan, Rui Yu, John Zhang +15
Agents operating in complex software environments benefit from reasoning about the consequences of their actions, as even a single incorrect user interface (UI) operation can derai…
G-KV: Decoding-Time KV Cache Eviction with Global Attention
Mengqi Liao, Lu Wang, Chaoyun Zhang +7
Recent reasoning large language models (LLMs) excel in complex tasks but encounter significant computational and memory challenges due to long sequence lengths. KV cache compressio…
GUI-360: A Comprehensive Dataset and Benchmark for Computer-Using Agents
Jian Mu, Chaoyun Zhang, Chiming Ni +14
We introduce GUI-360, a large-scale, comprehensive dataset and benchmark suite designed to advance computer-using agents (CUAs). CUAs present unique challenges and is const…
UFO3: Weaving the Digital Agent Galaxy
Chaoyun Zhang, Liqun Li, He Huang +8
Large language model (LLM)-powered agents are transforming digital devices from passive tools into proactive intelligent collaborators. However, most existing frameworks remain con…
From Task to Tutorial: An Automated GUI Framework for Excel Tutorial Document and Video Creation
Yuhang Xie, Jian Mu, Xiaojun Ma +9
Excel is one of the most widely used productivity tools across domains, offering rich functionality but also overwhelming users with its complexity. This creates a persistent deman…
GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents
Qianhui Wu, Kanzhi Cheng, Rui Yang +15
One of the principal challenges in building VLM-powered GUI agents is visual grounding, i.e., localizing the appropriate screen region for action execution based on both the visual…
Text2Grad: Reinforcement Learning from Natural Language Feedback
Hanyang Wang, Lu Wang, Chaoyun Zhang +5
Traditional RLHF optimizes language models with coarse, scalar rewards that mask the fine-grained reasons behind success or failure, leading to slow and opaque learning. Recent wor…
UFO2: The Desktop AgentOS
Chaoyun Zhang, He Huang, Chiming Ni +18
Recent Computer-Using Agents (CUAs), powered by multimodal large language models (LLMs), offer a promising direction for automating complex desktop workflows through natural langua…
API Agents vs. GUI Agents: Divergence and Convergence
Chaoyun Zhang, Shilin He, Liqun Li +5
Large language models (LLMs) have evolved beyond simple text generation to power software agents that directly translate natural language commands into tangible actions. While API-…
Large Action Models: From Inception to Implementation
Lu Wang, Fangkai Yang, Chaoyun Zhang +15
As AI continues to advance, there is a growing demand for systems that go beyond language-based assistance and move toward intelligent agents capable of performing real-world actio…
RuAG: Learned-rule-augmented Generation for Large Language Models
Yudi Zhang, Pei Xiao, Lu Wang +11
In-context learning (ICL) and Retrieval-Augmented Generation (RAG) have gained attention for their ability to enhance LLMs' reasoning by incorporating external knowledge but suffer…
Large Language Model-Brained GUI Agents: A Survey
Chaoyun Zhang, Shilin He, Jiaxu Qian +10
GUIs have long been central to human-computer interaction, providing an intuitive and visually-driven way to access and interact with digital systems. The advent of LLMs, particula…
Deoxys: A Causal Inference Engine for Unhealthy Node Mitigation in Large-scale Cloud Infrastructure
Chaoyun Zhang, Randolph Yao, Si Qin +8
The presence of unhealthy nodes in cloud infrastructure signals the potential failure of machines, which can significantly impact the availability and reliability of cloud services…
An Advanced Reinforcement Learning Framework for Online Scheduling of Deferrable Workloads in Cloud Computing
Hang Dong, Liwen Zhu, Zhao Shan +10
Efficient resource utilization and perfect user experience usually conflict with each other in cloud computing platforms. Great efforts have been invested in increasing resource ut…
Large Language Models can Deliver Accurate and Interpretable Time Series Anomaly Detection
Jun Liu, Chaoyun Zhang, Jiaxu Qian +6
Time series anomaly detection (TSAD) plays a crucial role in various industries by identifying atypical patterns that deviate from standard trends, thereby maintaining system integ…
Verco: Learning Coordinated Verbal Communication for Multi-agent Reinforcement Learning
Dapeng Li, Hang Dong, Lu Wang +8
In recent years, multi-agent reinforcement learning algorithms have made significant advancements in diverse gaming environments, leading to increased interest in the broader appli…
AllHands: Ask Me Anything on Large-scale Verbatim Feedback via Large Language Models
Chaoyun Zhang, Zicheng Ma, Yuhao Wu +12
Verbatim feedback constitutes a valuable repository of user experiences, opinions, and requirements essential for software development. Effectively and efficiently extracting valua…
UFO: A UI-Focused Agent for Windows OS Interaction
Chaoyun Zhang, Liqun Li, Shilin He +9
We introduce UFO, an innovative UI-Focused agent to fulfill user requests tailored to applications on Windows OS, harnessing the capabilities of GPT-Vision. UFO employs a dual-agen…
COIN: Chance-Constrained Imitation Learning for Uncertainty-aware Adaptive Resource Oversubscription Policy
Lu Wang, Mayukh Das, Fangkai Yang +9
We address the challenge of learning safe and robust decision policies in presence of uncertainty in context of the real scientific problem of adaptive resource oversubscription to…
Risk-aware Adaptive Virtual CPU Oversubscription in Microsoft Cloud via Prototypical Human-in-the-loop Imitation Learning
Lu Wang, Mayukh Das, Fangkai Yang +11
Oversubscription is a prevalent practice in cloud services where the system offers more virtual resources, such as virtual cores in virtual machines, to users or applications than…
Xpert: Empowering Incident Management with Query Recommendations via Large Language Models
Yuxuan Jiang, Chaoyun Zhang, Shilin He +8
Large-scale cloud systems play a pivotal role in modern IT infrastructure. However, incidents occurring within these systems can lead to service disruptions and adversely affect us…
Counter-Empirical Attacking based on Adversarial Reinforcement Learning for Time-Relevant Scoring System
Xiangguo Sun, Hong Cheng, Hang Dong +3
Scoring systems are commonly seen for platforms in the era of big data. From credit scoring systems in financial services to membership scores in E-commerce shopping platforms, pla…
TaskWeaver: A Code-First Agent Framework
Bo Qiao, Liqun Li, Xu Zhang +16
Large Language Models (LLMs) have shown impressive abilities in natural language understanding and generation, leading to their widespread use in applications such as chatbots and…