5 papers
How Far Are We from Detecting Flaky Tests? On the Limits of Code-Based Detection
Ãmer Oktay Gültekin, Alexander Berndt, Jonathan Bell +2
Flaky tests pass and fail on the same code version, weakening the signal of test results and disrupting continuous integration (CI) pipelines. Code-based flakiness detectors report…
Using Large Language Models to Support Automation of Failure Management in CI/CD Pipelines: A Case Study in SAP HANA
Duong Bui, Stefan Grintz, Alexander Berndt +1
CI/CD pipeline failure management is time-consuming when performed manually. Automating this process is non-trivial because the information required for effective failure managemen…
Can We Classify Flaky Tests Using Only Test Code? An LLM-Based Empirical Study
Alexander Berndt, Vekil Bekmyradov, Rainer Gemulla +3
Flaky tests yield inconsistent results when they are repeatedly executed on the same code revision. They interfere with automated quality assurance of code changes and hinder effic…
Flaky Tests in a Large Industrial Database Management System: An Empirical Study of Fixed Issue Reports for SAP HANA
Alexander Berndt, Thomas Bach, Sebastian Baltes
Flaky tests yield different results when executed multiple times for the same version of the source code. Thus, they provide an ambiguous signal about the quality of the code and i…
On the Flakiness of LLM-Generated Tests for Industrial and Open-Source Database Management Systems
Alexander Berndt, Thomas Bach, Rainer Gemulla +2
Flaky tests are a common problem in software testing. They produce inconsistent results when executed multiple times on the same code, invalidating the assumption that a test failu…