Validity, Reliability, and Transparency in Artificial Intelligence Regulation

Abstract

AI systems increasingly produce claims about people that shape access to employment, healthcare, and other consequential services. Yet lawful data processing and predictive accuracy do not establish that these claims justify the treatment that follows. We argue that regulation must therefore examine the legitimacy of the inferential step connecting data to claims and claims to decisions. This requires construct, internal, and external validity: evidence must support the meaning attributed to an output, the relationship asserted, and its application to the intended people and setting. We show why predictive performance alone cannot provide this warrant. Non-identifiability limits what observations can explain, while omissions and confounding require domain-specific judgment about what the evidence supports. Building on validity provisions in the EU AI Act and NIST AI RMF Playbook, we develop an explicit evidentiary burden for consequential reliance. Its normative basis follows from the dignity, autonomy, and informational-privacy principles in \emph{Puttaswamy} \citep{Puttaswamy2017Privacy}, which we extend from scrutiny of information acquisition to the justification of derived claims and their use. On this account, benefits that depend on an inference can carry weight in proportionality assessment only to the extent that the inference is substantiated. We make this requirement operational through a claim-specific \emph{Validity Case} linking evidence and assumptions to permitted uses, independent review, monitoring, and remedies. Hiring and LLM applications in healthcare and legal assistance illustrate how the framework can guide oversight of consequential public and private services through the relevant legal instruments.