Metamorphic Testing: A New Approach for Generating Next Test Cases

Abstract

In software testing, a set of test cases is constructed according to some predefined selection criteria. The software is then examined against these test cases. Three interesting observations have been made on the current artifacts of software testing. Firstly, an error-revealing test case is considered useful while a successful test case which does not reveal software errors is usually not further investigated. Whether these successful test cases still contain useful information for revealing software errors has not been properly studied. Secondly, no matter how extensive the testing has been conducted in the development phase, errors may still exist in the software [5]. These errors, if left undetected, may eventually cause damage to the production system. The study of techniques for uncovering software errors in the production phase is seldom addressed in the literature. Thirdly, as indicated by Weyuker in [6], the availability of test oracles is pragmatically unattainable in most situations. However, the availability of test oracles is generally assumed in conventional software testing techniques. In this paper, we propose a novel test case selection technique that derives new test cases from the successful ones. The selection aims at revealing software errors that are possibly left undetected in successful test cases which may be generated using some existing strategies. As such, the proposed technique augments the effectiveness of existing test selection strategies. The technique also helps uncover software errors in the production phase and can be used in the absence of test oracles.

11 pages, Technical Report HKUST-CS98-01, Department of Computer Science, The Hong Kong University of Science and Technology

Cited by in corpus (8)

• Testing Database Engines via Pivoted Query Synthesis
• Metamorphic Testing for Object Detection Systems
• Metamorphic Detection of Adversarial Examples in Deep Learning Models With Affine Transformations
• Challenges for Verifying and Validating Scientific Software in Computational Materials Science
• Automatic Testing and Improvement of Machine Translation
• Object-based Metamorphic Testing through Image Structuring
• Towards Structured Evaluation of Deep Neural Network Supervisors
• Metamorphic Testing for Quality Assurance of Protein Function Prediction Tools