6 citations · 10 across the 6 of their papers we have counts for
7 papers
Who Finishes the Job? A Study of Follow-Up Fixes and Commit Authorship on AI Coding Agent Pull Requests
Wannita Takerngsaksiri, Nhat Duong, Scott Barnett
AI coding agents now author a large share of pull requests (PRs) merged into popular open-source projects. A merged agent PR is usually considered finished work; yet, prior studies…
TaskEval: Synthesised Evaluation for Foundation-Model Tasks
Dilani Widanapathiranage, Scott Barnett, Stefanus Kurniawan +1
Hallucinations are a key concern when creating applications that rely on Foundation models (FMs). Understanding where and how these subtle failures occur in an application relies o…
The M-factor: A Novel Metric for Evaluating Neural Architecture Search in Resource-Constrained Environments
Srikanth Thudumu, Hy Nguyen, Hung Du +6
Neural Architecture Search (NAS) aims to automate the design of deep neural networks. However, existing NAS techniques often focus on maximising accuracy, neglecting model efficien…
RAGProbe: An Automated Approach for Evaluating RAG Applications
Shangeetha Sivasothy, Scott Barnett, Stefanus Kurniawan +2
Retrieval Augmented Generation (RAG) is increasingly being used when building Generative AI applications. Evaluating these applications and RAG pipelines is mostly done manually, v…
Fine-Tuning or Fine-Failing? Debunking Performance Myths in Large Language Models
Scott Barnett, Zac Brannelly, Stefanus Kurniawan +1
Large Language Models (LLMs) have the unique capability to understand and generate human-like text from input queries. When fine-tuned, these models show enhanced performance on do…
Large language models for generating rules, yay or nay?
Shangeetha Sivasothy, Scott Barnett, Rena Logothetis +4
Engineering safety-critical systems such as medical devices and digital health intervention systems is complex, where long-term engagement with subject-matter experts (SMEs) is nee…
LLMs for Test Input Generation for Semantic Caches
Zafaryab Rasool, Scott Barnett, David Willie +4
Large language models (LLMs) enable state-of-the-art semantic capabilities to be added to software systems such as semantic search of unstructured documents and text generation. Ho…