3 citations · 5 across the 16 of their papers we have counts for
11 papers · 1 filter
AdaT: Adaptive Test Transformations for Black-Box Boundary Testing of Conversational Agents
Liting Lin, Boxi Yu, Qinghua Xu +3
Conversational agents based on large language models (LLMs) must comply with policies. Each condition in a policy draws a boundary between user requests, and the agent must behave…
DreamTest: World-Model Surrogates for Search-Based Testing of Deep Reinforcement Learning Agents
Qinghua Xu, Guancheng Wang, Boxi Yu +2
Testing deep reinforcement learning (DRL) agents in cyber-physical systems aims to uncover diverse failures before deployment, but each execution can be expensive. Surrogate-assist…
Mining Workflow Graphs for Black-Box Boundary Testing of Conversational LLM Agents
Liting Lin, Boxi Yu, Yuzhong Zhang +3
Conversational LLM agents can cause real-world harm when their internal workflows fail, such as completing a transaction without confirmation. Testing these state-dependent failure…
BeSpec: Behavior-Level Specification Alignment for Code Generation
Qinghua Xu, Guancheng Wang, Boxi Yu +1
LLMs have made substantial progress on automated code generation from natural-language descriptions of desired behavior (intent). Most existing methods improve generated programs t…
SWE-ABS: Adversarial Benchmark Strengthening Exposes Inflated Success Rates on Test-based Benchmark
Boxi Yu, Yang Cao, Yuzhong Zhang +9
The SWE-Bench Verified leaderboard is approaching saturation, with the top system achieving 78.80%. However, we show that this performance is inflated. Our re-evaluation reveals th…
UTBoost: Rigorous Evaluation of Coding Agents on SWE-Bench
Boxi Yu, Yuxuan Zhu, Pinjia He +1
The advent of Large Language Models (LLMs) has spurred the development of coding agents for real-world code generation. As a widely used benchmark for evaluating the code generatio…
Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility
Jialun Cao, Yuk-Kit Chan, Zixuan Ling +12
Code-related benchmarks play a critical role in evaluating large language models (LLMs), yet their quality fundamentally shapes how the community interprets model capabilities. In…
An Empirical Study on Package-Level Deprecation in Python Ecosystem
Zhiqing Zhong, Shilin He, Haoxuan Wang +3
Open-source software (OSS) plays a crucial role in modern software development. Utilizing OSS code can greatly accelerate software development, reduce redundancy, and enhance relia…
Retromorphic Testing: A New Approach to the Test Oracle Problem
Boxi Yu, Qiuyang Mang, Qingshuo Guo +1
A test oracle serves as a criterion or mechanism to assess the correspondence between software output and the anticipated behavior for a given input set. In automated testing, blac…
ROME: Testing Image Captioning Systems via Recursive Object Melting
Boxi Yu, Zhiqing Zhong, Jiaqi Li +3
Image captioning (IC) systems aim to generate a text description of the salient objects in an image. In recent years, IC systems have been increasingly integrated into our daily li…
Automated Testing of Image Captioning Systems
Boxi Yu, Zhiqing Zhong, Xinran Qin +3
Image captioning (IC) systems, which automatically generate a text description of the salient objects in an image (real or synthetic), have seen great progress over the past few ye…