465 citations · 992 across the 50 of their papers we have counts for
52 papers
ModelWisdom: An Integrated Toolkit for TLA+ Model Visualization, Digest and Repair
Zhiyong Chen, Jialun Cao, Chang Xu +1
Model checking in TLA+ provides strong correctness guarantees, yet practitioners continue to face significant challenges in interpreting counterexamples, understanding large state-…
RulER: Automated Rule-Based Semantic Error Localization and Repair for Code Translation
Shuo Jin, Songqiang Chen, Xiaoyuan Xie +1
Automated code translation aims to convert programs between different programming languages while maintaining their functionality. Due to the imperfections of code translation mode…
What Makes In-Context Examples Effective for Code Generation?
Dongze Li, Songqiang Chen, Jialun Cao +1
In-Context Learning (ICL) has emerged as a promising solution to enhance the code generation capabilities of Large Language Models (LLMs) by incorporating code examples inside the…
EmbedAgent: Benchmarking Large Language Models in Embedded System Development
Ruiyang Xu, Jialun Cao, Mingyuan Wu +6
Large Language Models (LLMs) have shown promise in various tasks, yet few benchmarks assess their capabilities in embedded system development. In this paper, we introduce EmbedAgen…
Across Programming Language Silos: A Study on Cross-Lingual Retrieval-augmented Code Generation
Qiming Zhu, Jialun Cao, Xuanang Chen +6
Current research on large language models (LLMs) with retrieval-augmented code generation (RACG) has largely focused on single-language settings, leaving their cross-lingual effect…
ReuseDroid: A VLM-empowered Android UI Test Migrator Boosted by Active Feedback
Xiaolei Li, Jialun Cao, Yepang Liu +2
GUI testing is an essential quality assurance process in mobile app development. However, the creation and maintenance of GUI tests for mobile apps are resource-intensive and costl…
When LLMs Meet API Documentation: Can Retrieval Augmentation Aid Code Generation Just as It Helps Developers?
Jingyi Chen, Songqiang Chen, Jialun Cao +2
Retrieval-augmented generation (RAG) has increasingly shown its power in extending large language models' (LLMs') capability beyond their pre-trained knowledge. Existing works have…
Isolating Language-Coding from Problem-Solving: Benchmarking LLMs with PseudoEval
Jiarong Wu, Songqiang Chen, Jialun Cao +2
Existing code generation benchmarks for Large Language Models (LLMs) such as HumanEval and MBPP are designed to study LLMs' end-to-end performance, where the benchmarks feed a prob…
How Far are App Secrets from Being Stolen? A Case Study on Android
Lili Wei, Heqing Huang, Shing-Chi Cheung +1
Android apps can hold secret strings of themselves such as cloud service credentials or encryption keys. Leakage of such secret strings can induce unprecedented consequences like m…
From Informal to Formal -- Incorporating and Evaluating LLMs on Natural Language Requirements to Verifiable Formal Proofs
Jialun Cao, Yaojie Lu, Meiziniu Li +9
The research in AI-based formal mathematical reasoning has shown an unstoppable growth trend. These studies have excelled in mathematical competitions like IMO and have made signif…
Code Benchmarks Should Prioritize Rigor, Reliability, and Reproducibility
Jialun Cao, Yuk-Kit Chan, Zixuan Ling +12
Code-related benchmarks play a critical role in evaluating large language models (LLMs), yet their quality fundamentally shapes how the community interprets model capabilities. In…
CODECLEANER: Elevating Standards with A Robust Data Contamination Mitigation Toolkit
Jialun Cao, Songqiang Chen, Wuqi Zhang +2
Data contamination presents a critical barrier preventing widespread industrial adoption of advanced software engineering techniques that leverage code language models (CLMs). This…
DOMAINEVAL: An Auto-Constructed Benchmark for Multi-Domain Code Generation
Qiming Zhu, Jialun Cao, Yaojie Lu +4
Code benchmarks such as HumanEval are widely adopted to evaluate the capabilities of Large Language Models (LLMs), providing insights into their strengths and weaknesses. However,…
A Tale of Two DL Cities: When Library Tests Meet Compiler
Qingchao Shen, Yongqiang Tian, Haoyang Ma +5
Deep Learning (DL) compilers typically load a DL model and optimize it with intermediate representation.Existing DL compiler testing techniques mainly focus on model optimization s…
Towards Understanding the Bugs in Solidity Compiler
Haoyang Ma, Wuqi Zhang, Qingchao Shen +3
Solidity compiler plays a key role in enabling the development of smart contract applications on Ethereum by governing the syntax of a domain-specific language called Solidity and…
MR-Adopt: Automatic Deduction of Input Transformation Function for Metamorphic Testing
Congying Xu, Songqiang Chen, Jiarong Wu +4
While a recent study reveals that many developer-written test cases can encode a reusable Metamorphic Relation (MR), over 70% of them directly hard-code the source input and follow…
CRUXEval-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution
Ruiyang Xu, Jialun Cao, Yaojie Lu +6
Code benchmarks such as HumanEval are widely adopted to evaluate Large Language Models' (LLMs) coding capabilities. However, there is an unignorable programming language bias in ex…
JavaBench: A Benchmark of Object-Oriented Code Generation for Evaluating Large Language Models
Jialun Cao, Zhiyong Chen, Jiarong Wu +2
Code generation benchmarks such as HumanEval are widely adopted to evaluate LLMs' capabilities. However, after consolidating the latest 24 benchmarks, we noticed three significant…
Enhancing Differential Testing With LLMs For Testing Deep Learning Libraries
Meiziniu Li, Dongze Li, Jianmeng Liu +3
Differential testing offers a promising strategy to alleviate the test oracle problem by comparing the test results between alternative implementations. However, existing different…
Automatic Build Repair for Test Cases using Incompatible Java Versions
Ching Hang Mak, Shing-Chi Cheung
Context: Bug bisection is a common technique used to identify a revision that introduces a bug or indirectly fixes a bug, and often involves executing multiple revisions of a proje…
Enchanting Program Specification Synthesis by Large Language Models using Static Analysis and Program Verification
Cheng Wen, Jialun Cao, Jie Su +6
Formal verification provides a rigorous and systematic approach to ensure the correctness and reliability of software systems. Yet, constructing specifications for the full proof r…
Concerned with Data Contamination? Assessing Countermeasures in Code Language Model
Jialun Cao, Wuqi Zhang, Shing-Chi Cheung
Various techniques have been proposed to leverage the capabilities of code language models (CLMs) for SE tasks. While these techniques typically evaluate their effectiveness using…
Word Closure-Based Metamorphic Testing for Machine Translation
Xiaoyuan Xie, Shuo Jin, Songqiang Chen +1
With the wide application of machine translation, the testing of Machine Translation Systems (MTSs) has attracted much attention. Recent works apply Metamorphic Testing (MT) to add…
Modeling Software Quality in Virtual Reality Applications from User Feedback
Shuqing Li, Chenran Zhang, Yepang Liu +3
Virtual Reality (VR) applications couple software behavior with head-mounted displays, body movement, spatial interaction, and multimodal feedback. Consequently, familiar software-…