5 papers · 1 filter
OpenHarmony Bench: Evaluating LLMs and Coding Agents on OpenHarmony App Development
Li Li, Han Hu, Tianjian Zhang +27
We present OPENHARMONY BENCH, an app-level coding benchmark for evaluating LLM-based coding agents on OpenHarmony ArkTS applications. Unlike function-level benchmarks, it evaluates…
Rover: Context-aware Conflict Resolution with LLM
Qingyu Zhang, Junzhe Li, Jiayi Lin +2
Code merging is a significant challenge, particularly in large-scale projects. Existing solutions, including program analysis and machine learning, show promise but face critical l…
CodeFuse-CommitEval: Towards Benchmarking LLM's Power on Commit Message and Code Change Inconsistency Detection
Qingyu Zhang, Puzhuo Liu, Peng Di +1
Version control relies on commit messages to convey the rationale for code changes, but these messages are often low quality and, more critically, inconsistent with their diffs-kno…
CONGRA: Benchmarking Automatic Conflict Resolution
Qingyu Zhang, Liangcai Su, Kai Ye +1
Resolving conflicts from merging different software versions is a challenging task. To reduce the overhead of manual merging, researchers develop various program analysis-based too…
WizardMerge -- Save Us From Merging Without Any Clues
Qingyu Zhang, Junzhe Li, Jiayi Lin +3
Modern software development necessitates efficient version-oriented collaboration among developers. While Git is the most popular version control system, it generates unsatisfactor…