5 papers · 1 filter
Structural Anchors and Reasoning Fragility:Understanding CoT Robustness in LLM4Code
Yang Liu, Da Song, Armstrong Foundjem +2
Chain-of-Thought (CoT) prompting is widely used to elicit explicit reasoning from large language models for code (LLM4Code). However, its impact on robustness and the stability of…
Improving the Robustness of Large Language Models for Code Tasks via Fine-tuning with Perturbed Data
Yang Liu, Armstrong Foundjem, Xingfang Wu +2
Context: In the fast-paced evolution of software development, Large Language Models (LLMs) have become indispensable tools for tasks such as code generation, completion, analysis,…
Risk Management for Mitigating Benchmark Failure Modes: BenchRisk
Sean McGregor, Victor Lu, Vassil Tashev +8
Large language model (LLM) benchmarks inform LLM use decisions (e.g., "is this LLM safe to deploy for my use case and context?"). However, benchmarks may be rendered unreliable by…
Adversarial Attack Classification and Robustness Testing for Large Language Models for Code
Yang Liu, Armstrong Foundjem, Foutse Khomh +1
Large Language Models (LLMs) have become vital tools in software development tasks such as code generation, completion, and analysis. As their integration into workflows deepens, e…
An empirical study of testing machine learning in the wild
Moses Openja, Foutse Khomh, Armstrong Foundjem +4
Recently, machine and deep learning (ML/DL) algorithms have been increasingly adopted in many software systems. Due to their inductive nature, ensuring the quality of these systems…