4 papers
Operational Robustness of LLMs on Code Generation
Debalina Ghosh Paul, Hong Zhu, Ian Bayley
It is now common practice in software development for large language models (LLMs) to be used to generate program code. It is desirable to evaluate the robustness of LLMs for this…
MASTEST: A LLM-Based Multi-Agent System For RESTful API Tests
Xiaoke Han, Hong Zhu
Testing RESTful API is increasingly important in quality assurance of cloud-native applications. Recent advances in machine learning (ML) techniques have demonstrated that various…
Investigating The Smells of LLM Generated Code
Debalina Ghosh Paul, Hong Zhu, Ian Bayley
Context: Large Language Models (LLMs) are increasingly being used to generate program code. Much research has been reported on the functional correctness of generated code, but the…
Exploring Performance Contrasts in TableQA: Step-by-Step Reasoning Boosts Bigger Language Models, Limits Smaller Language Models
Haoyan Yang, Yixuan Wang, Keyue Tong +2
This paper proposes a detailed prompting flow, termed Table-Logic, to investigate the performance contrasts between bigger and smaller language models (LMs) utilizing step-by-step…