Data Validation

Abstract

Data validation is the activity where one decides whether or not a particular data set is fit for a given purpose. Formalizing the requirements that drive this decision process allows for unambiguous communication of the requirements, automation of the decision process, and opens up ways to maintain and investigate the decision process itself. The purpose of this article is to formalize the definition of data validation and to demonstrate some of the properties that can be derived from this definition. In particular, it is shown how a formal view of the concept permits a classification of data quality requirements, allowing them to be ordered in increasing levels of complexity. Some subtleties arising from combining possibly many such requirements are pointed out as well.

arXiv comment7 pages, 1 figure. In Wiley StatsRef: Statistics Reference Online (2020)

References in corpus (1)

• Data Validation Infrastructure for R

Cited by in corpus (13)

• The electron mass from $g$-factor measurements on hydrogen-like carbon ${}^{12}$C${}^{5+}$
• Predicting fuel research octane number using Fourier-transform infrared absorption spectra of neat hydrocarbons
• Search for an anomalous excess of charged-current quasi-elastic ${\nu }_e$ interactions with the MicroBooNE experiment using Deep-Learning-based reconstruction
• Tournament design: A review from an operational research perspective
• Biwhitening Reveals the Rank of a Count Matrix
• Percolation thresholds on triangular lattice for neighbourhoods containing sites up-to the fifth coordination zone
• Stochastic analysis of the Elo rating algorithm in round-robin tournaments
• Numerical simulations of atmospheric quantum channels
• Voicify Your UI: Towards Android App Control with Voice Commands
• Using Ripley's K-function to Characterize Clustering In 3-Dimensional Point Patterns With a Case Study in Atom Probe Tomography
• An unsupervised machine-learning checkpoint-restart algorithm using Gaussian mixtures for particle-in-cell simulations
• Random site percolation thresholds on square lattice for complex neighborhoods containing sites up to the sixth coordination zone
• Optimal Design of Validation Experiments for the Prediction of Quantities of Interest