13 papers
One QK Channel, Many Sources: Guarding Low-Precision Attention Collapse
Shuxiao Xie, Shuyang Xie, Yuan Cao +3
A bfloat16 transformer can train normally for many steps and then collapse abruptly. Distinct low-precision errors can trigger the same failure, leaving unclear whether each source…
BackendForge: Benchmarking Agentic End-to-End Code Generation with Backend Services
Yuzhe Guo, Mengzhou Wu, Yuan Cao +4
The paper presents BackendForge, a benchmark that evaluates how well agentic large language models can generate complete, deployable backend services from OpenAPI specifications, u…
An Empirical Study and Theoretical Explanation on Task-Level Model-Merging Collapse
Yuan Cao, Dezhi Ran, Yuzhe Guo +5
Model merging unifies independently fine-tuned LLMs from the same base, enabling reuse and integration of parallel development efforts without retraining. However, in practice we o…
GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training
Yuan Cao, Dezhi Ran, Mengzhou Wu +9
Post-training GUI agents in interactive environments is critical for developing generalization and long-horizon planning capabilities. However, training on real-world applications…
UI-Oceanus: Scaling GUI Agents with Synthetic Environmental Dynamics
Mengzhou Wu, Yuzhe Guo, Yuan Cao +16
Scaling generalist GUI agents is hindered by the data scalability bottleneck of expensive human demonstrations and the "distillation ceiling" of synthetic teacher supervision. To t…
KernelBand: Steering LLM-based Kernel Optimization via Hardware-Aware Multi-Armed Bandits
Dezhi Ran, Shuxiao Xie, Mingfang Ji +9
High-performance GPU kernels are critical for efficient LLM serving, yet their optimization remains a bottleneck requiring deep system expertise. While code LLMs show promise in ge…