87 citations · 168 across the 51 of their papers we have counts for
18 papers · 1 filter
When Agent Evaluation Becomes Unreliable: An Empirical Study of Oracle Failures and Configuration-Dependent Measurement in an LLM Agent Benchmark
Junjie Chen
Agent evaluation harnesses are software: they run a test suite, drive a test input generator, and decide outcomes with a test oracle. We audit them for defects a score table cannot…
Learning Globally Reusable Skills for Coding Agents
Chen Yang, Jiashuo Tian, Ziqi Wang +3
Automated skill evolution enables Large Language Model (LLM) agents to continuously improve without expensive retraining. However, existing approaches typically treat skill evoluti…
Agent-Based Test Assertion Generation via Diverse Perspective Aggregation
Dong Wang, Qiaoyu Han, Lin Yang +3
Test assertions are critical elements of unit tests, serving as checkpoints to validate expected behavior and ensure software correctness. Numerous techniques have been proposed to…
Evaluating Fuzz Testing for Reinforcement Learning Agents
Zhibin Kang, Hanmo You, Dong Wang +2
Reinforcement Learning (RL) agents are increasingly deployed in safety-critical domains such as robotics, autonomous driving, and drone control, where unexpected behaviors may lead…
KernelDiag: Agent-Based Root Cause Diagnosis for Kernel Crashes
Weijing Wang, Zan Wang, Dong Wang +2
The Linux kernel is one of the most complex software systems, where automated fuzzing continuously exposes thousands of crashes, yet root-cause diagnosis remains a manual and time-…
GapForge: Directed Compiler Fuzzing via Coverage-Gap Analysis
Mingxuan Zhu, Qingyuan Liang, Junjie Chen +2
Modern compiler codebases (e.g., GCC and LLVM) are large and complex, making comprehensive coverage across diverse code regions highly challenging. Most existing test generation te…
LLVM-Bench: Benchmarking and Advancing Large Language Models for LLVM Compiler Issue Resolution
Zhao Tian, Yingquan Zhao, Chenyao Suo +2
LLVM is a widely used compiler infrastructure whose scale and complexity make issue resolution labor-intensive and challenging. Although large language models (LLMs) have recently…
Towards Better Linux Kernel Fault Localization: Leveraging Contrastive Reasoning and Hierarchical Context Analysis
Haichi Wang, Ruiguo Yu, Yesong Pang +4
Debugging the Linux kernel remains a formidable challenge due to its vast codebase, complex architecture, and low-level programming intricacies. Effective fault localization (FL) i…
Large Language Models for Multi-Lingual Equivalent Mutant Detection: An Extended Empirical Study
Honglin Shu, Zhao Tian, Dong Wang +5
Mutation testing is a powerful technique for ensuring software quality. However, the presence of equivalent mutants introduces unnecessary costs and biases, limiting its practical…
"Refactoring Runaway": Understanding and Mitigating Tangled Refactorings in Coding Agents for Issue Resolution
Zhao Tian, Zifan Zhang, Tao Xiao +4
Recent advances in coding agents have shown remarkable progress in software issue resolution. In practice, real-world issues are typically bug fixes or feature requests in which hu…
Characterizing the Failure Modes of LLMs in Resolving Real-World GitHub Issues
Yanjie Jiang, Yian Huang, Guancheng Wang +3
Large Language Models (LLMs) are increasingly deployed to resolve real-world GitHub issues. However, despite their potential, the specific failure modes of these models in complex…
Characterizing and Mitigating False-Positive Bug Reports in the Linux Kernel
Jiashuo Tian, Dong Wang, Chen Yang +3
False-positive bug reports represent a significant yet underexplored challenge in the development and maintenance of the Linux kernel. They occur when correct system behavior is mi…
Improving LLM Code Generation via Requirement-Aware Curriculum Reinforcement Learning
Shouyu Yin, Zhao Tian, Junjie Chen +1
Code generation, which aims to automatically generate source code from given programming requirements, has the potential to substantially improve software development efficiency. W…
TEMPLATEFUZZ: Fine-Grained Chat Template Fuzzing for Jailbreaking and Red Teaming LLMs
Qingchao Shen, Zibo Xiao, Lili Huang +3
Large Language Models (LLMs) are increasingly deployed across diverse domains, yet their vulnerability to jailbreak attacks, where adversarial inputs bypass safety mechanisms to el…
REAgent: Requirement-Driven LLM Agents for Software Issue Resolution
Shiqi Kuang, Zhao Tian, Kaiwei Lin +5
Issue resolution aims to automatically generate patches from given issue descriptions and has attracted significant attention with the rapid advancement of large language models (L…
Uncovering Business Logic Bugs via Semantics-Driven Unit Test Generation
Chen Yang, Junjie Chen
Business logic bugs violate intended business semantics and are particularly prevalent in enterprise software. Yet most existing unit test generation techniques are code-centric, m…
Forecasting Developer Environments with GenAI: A Research Perspective
Raula Gaikovina Kula, Christoph Treude, Xing Hu +28
Generative Artificial Intelligence (GenAI) models are achieving remarkable performance in various tasks, including code generation, testing, code review, and program repair. The ab…
AIR: Improving Agent Safety through Incident Response
Zibo Xiao, Jun Sun, Junjie Chen
Large Language Model (LLM) agents are increasingly deployed in practice across a wide range of autonomous applications. Yet current safety mechanisms for LLM agents focus almost ex…