6 papers
OpenHarmony Bench: Evaluating LLMs and Coding Agents on OpenHarmony App Development
Li Li, Han Hu, Tianjian Zhang +27
We present OPENHARMONY BENCH, an app-level coding benchmark for evaluating LLM-based coding agents on OpenHarmony ArkTS applications. Unlike function-level benchmarks, it evaluates…
Rover: Context-aware Conflict Resolution with LLM
Qingyu Zhang, Junzhe Li, Jiayi Lin +2
Code merging is a significant challenge, particularly in large-scale projects. Existing solutions, including program analysis and machine learning, show promise but face critical l…
CodeFuse-CommitEval: Towards Benchmarking LLM's Power on Commit Message and Code Change Inconsistency Detection
Qingyu Zhang, Puzhuo Liu, Peng Di +1
Version control relies on commit messages to convey the rationale for code changes, but these messages are often low quality and, more critically, inconsistent with their diffs-kno…
CONGRA: Benchmarking Automatic Conflict Resolution
Qingyu Zhang, Liangcai Su, Kai Ye +1
Resolving conflicts from merging different software versions is a challenging task. To reduce the overhead of manual merging, researchers develop various program analysis-based too…
Identify As A Human Does: A Pathfinder of Next-Generation Anti-Cheat Framework for First-Person Shooter Games
Jiayi Zhang, Chenxin Sun, Yue Gu +4
The gaming industry has experienced substantial growth, but cheating in online games poses a significant threat to the integrity of the gaming experience. Cheating, particularly in…
WizardMerge -- Save Us From Merging Without Any Clues
Qingyu Zhang, Junzhe Li, Jiayi Lin +3
Modern software development necessitates efficient version-oriented collaboration among developers. While Git is the most popular version control system, it generates unsatisfactor…