8 papers
Can Large Language Models Generate Observability-Aware Code?
Yongliang Tao, Hongyu Zhang, Pengfei Gao +8
Recent advances in coding agents have enabled the generation of increasingly complex software systems. While existing evaluations primarily focus on functional correctness, product…
UFO3: Weaving the Digital Agent Galaxy
Chaoyun Zhang, Liqun Li, He Huang +8
Large language model (LLM)-powered agents are transforming digital devices from passive tools into proactive intelligent collaborators. However, most existing frameworks remain con…
DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems
Ming Ma, Jue Zhang, Fangkai Yang +4
Large language model (LLM)-based multi-agent systems are challenging to debug because failures often arise from long, branching interaction traces. The prevailing practice is to le…
GUI-360: A Comprehensive Dataset and Benchmark for Computer-Using Agents
Jian Mu, Chaoyun Zhang, Chiming Ni +14
We introduce GUI-360, a large-scale, comprehensive dataset and benchmark suite designed to advance computer-using agents (CUAs). CUAs present unique challenges and is const…
An Empirical Study of Production Incidents in Generative AI Cloud Services
Haoran Yan, Yinfang Chen, Minghua Ma +10
The ever-increasing demand for generative artificial intelligence (GenAI) has motivated cloud-based GenAI services such as Azure OpenAI Service and Amazon Bedrock. Like any large-s…
Large Language Model-Brained GUI Agents: A Survey
Chaoyun Zhang, Shilin He, Jiaxu Qian +10
GUIs have long been central to human-computer interaction, providing an intuitive and visually-driven way to access and interact with digital systems. The advent of LLMs, particula…