3 citations · 3 across the 9 of their papers we have counts for
10 papers
Metrics That Write Themselves: Evolving an Evaluator from Its Own Blind Spots
Xing Zhang, Yanwei Cui, Guanghui Wang +2
Agents improve quickly against a reliable automatic metric and stall without one, and the applications that need them most, report generation among them, are the ones nobody knows…
OpenHarmony Bench: Evaluating LLMs and Coding Agents on OpenHarmony App Development
Li Li, Han Hu, Tianjian Zhang +27
We present OPENHARMONY BENCH, an app-level coding benchmark for evaluating LLM-based coding agents on OpenHarmony ArkTS applications. Unlike function-level benchmarks, it evaluates…
AtomicCommitBench: Can Coding Agents Reconstruct Commit Histories from Squashed Patches?
Zhihao Lin, Mingyi Zhou, Li Li
Coding agents often finish a session by returning one squashed patch that mixes feature implementation, bug fixes, refactorings, tests, and configuration edits. While the final cod…
How Much Static Structure Do Code Agents Need? A Study of Deterministic Anchoring
Zhihao Lin, Mingyi Zhou, Yizhuo Yang +1
LLM-based code agents navigate repositories through keyword search but miss the structural relationships, such as call graphs, inheritance hierarchies, and configuration dependenci…
RepoRescue: An Empirical Study of LLM Agents on Whole-Repository Compatibility Rescue
Zhihao Lin, Mingyi Zhou, Zhensu Sun +4
Open-source libraries and tools are widely reused, but compatibility maintenance is expensive. Once maintainers leave, useful repositories can stop working as runtimes and dependen…
To Run or Not to Run: Analyzing the Cost-Effectiveness of Code Execution in LLM-Based Program Repair
Zhihao Lin, Junhua Zhu, Mingyi Zhou +5
LLM-based agents for program repair are increasingly built on a "generate-run-revise" paradigm, iteratively executing tests to evaluate and refine patches. This execution-based app…
Executing as You Generate: Hiding Execution Latency in LLM Code Interpreters
Zhensu Sun, Zhihao Lin, Zhi Chen +4
Current LLM systems are increasingly equipped with a code interpreter that executes generated code to obtain results. This works serially: the model first generates the complete co…
PathSeeker: Exploring LLM Security Vulnerabilities with a Reinforcement Learning-Based Jailbreak Approach
Zhihao Lin, Wei Ma, Mingyi Zhou +5
In recent years, Large Language Models (LLMs) have gained widespread use, raising concerns about their security. Traditional jailbreak attacks, which often rely on the model intern…
Open-Source AI-based SE Tools: Opportunities and Challenges of Collaborative Software Learning
Zhihao Lin, Wei Ma, Tao Lin +7
Large Language Models (LLMs) have become instrumental in advancing software engineering (SE) tasks, showcasing their efficacy in code understanding and beyond. Like traditional SE…
Exploring Code Analysis: Zero-Shot Insights on Syntax and Semantics with LLMs
Wei Ma, Zhihao Lin, Shangqing Liu +8
Code analysis is fundamental in Software Engineering, supporting debugging, optimization, and security assessment. Human developers approach it through syntax parsing, static seman…