6 papers
From Reasoning Strings to Partial Orders: Verifier-Certified Rule Transport through Quotient Policy Optimization
Bang Xie, Hao Liu, Zhiyuan Peng +5
Many computations admit several valid execution orders because independent subgoals or disjoint state updates can commute. Reinforcement learning with verifiable rewards usually tr…
PonyEval: Evaluating LLM-Based Program Repair for Capability-Safe and Actor-Oriented Pony Software
Bang Xie, Hao Liu, Zhenyu Shi +8
Repository-level issue-resolution benchmarks have made executable evaluation central to software-engineering agents, but their language coverage remains concentrated in mainstream…
OdinEval: A Reproducible Benchmark for LLM-Based Program Repair in the Odin Programming Language
Bang Xie, Hao Liu, Zhiyuan Peng +8
Repository-level repair benchmarks still center on a few mainstream languages, leaving systems languages such as Odin largely untested. We present OdinEval, a reproducible benchmar…
AppEval: A Unified Benchmark for LLM-Based Mobile Application Repair in ArkTS, Swift, and Kotlin
Bang Xie, Hao Liu, Zhenyu Shi +11
Repository-level LLM agents are typically evaluated on projects whose tests run on the build host. It remains unclear whether their repairs survive the mobile build-install-launch-…
SafePyramid: A Hierarchical Benchmark for In-context Policy Guardrailing
Jiacheng Zhang, Haoyu He, Sen Zhang +5
In real-world applications, guardrails are often expected to identify unsafe user-model interactions according to application-specific safety policies, rather than relying on prede…
ArkEval: Benchmarking and Evaluating Automated CodeRepair for ArkTS
Bang Xie, Senjian Zhang, Zhiyuan Peng +3
Large language models have transformed code generation, enabling unprecedented automation in software development. As mobile ecosystems evolve, HarmonyOS has emerged as a critical…