9 papers · 1 filter
How Far Are We from Detecting Flaky Tests? On the Limits of Code-Based Detection
Ömer Oktay Gültekin, Alexander Berndt, Jonathan Bell +2
Flaky tests pass and fail on the same code version, weakening the signal of test results and disrupting continuous integration (CI) pipelines. Code-based flakiness detectors report…
Context Rot in AI-Assisted Software Development: Repurposing Documentation Consistency for AI Configuration Artifacts
Christoph Treude, Sebastian Baltes
Developers increasingly provide AI coding assistants with persistent context through configuration files such as CLAUDE.md, AGENTS.md, and .cursorrules. These files describe code e…
Can We Classify Flaky Tests Using Only Test Code? An LLM-Based Empirical Study
Alexander Berndt, Vekil Bekmyradov, Rainer Gemulla +3
Flaky tests yield inconsistent results when they are repeatedly executed on the same code revision. They interfere with automated quality assurance of code changes and hinder effic…
Flaky Tests in a Large Industrial Database Management System: An Empirical Study of Fixed Issue Reports for SAP HANA
Alexander Berndt, Thomas Bach, Sebastian Baltes
Flaky tests yield different results when executed multiple times for the same version of the source code. Thus, they provide an ambiguous signal about the quality of the code and i…
On the Flakiness of LLM-Generated Tests for Industrial and Open-Source Database Management Systems
Alexander Berndt, Thomas Bach, Rainer Gemulla +2
Flaky tests are a common problem in software testing. They produce inconsistent results when executed multiple times on the same code, invalidating the assumption that a test failu…
How Does Cognitive Capability and Personality Influence Problem Solving in Coding Interview Puzzles?
Dulaji Hidellaarachchi, Sebastian Baltes, John Grundy
Software engineering involves cognitively demanding activities impacted by individual differences. We investigate how cognitive capability and personality traits are associated wit…
The Influence of Code Comments on the Perceived Helpfulness of Stack Overflow Posts
Kathrin Figl, Maria Kirchner, Sebastian Baltes +1
Question-and-answer platforms such as Stack Overflow are an important way for software developers to share and retrieve knowledge. However, reusing poorly understood code can lead…
Do Test and Environmental Complexity Increase Flakiness? An Empirical Study of SAP HANA
Alexander Berndt, Thomas Bach, Sebastian Baltes
Background: Test flakiness is a major problem in the software industry. Flaky tests fail seemingly at random without changes to the code and thus impede continuous integration (CI)…
Taming Timeout Flakiness: An Empirical Study of SAP HANA
Alexander Berndt, Sebastian Baltes, Thomas Bach
Regression testing aims to prevent code changes from breaking existing features. Flaky tests negatively affect regression testing because they result in test failures that are not…