Measuring AI harms with multidimensional Lorenz Zonoids

Abstract

While AI systems increasingly shape high-stakes societal domains, their governance is limited by the lack of risk management methods that operate on real harms, taking their severity, and not only their likelihood, into account. As a consequence, AI risk management models remain compliance-driven and provider-centric, offering limited insight into how harms are dangerous, and on what should be the priority of intervention. The problem is amplified by the nature of harm data which are typically ordinal and multidimensional. To solve the problem, and offer an effective risk assessment methodology, in this paper we propose to model harm data by means of Lorenz Zonoids and Gini indices. To this aim we propose to extend them in a multidimensional setting, and show how to practically calculate them for a real AI incident data repository, provided by the Massachusetts Institute of Technology. The empirical findings indicate that environmental, infrastructure, property, physical, and democracy-related harms attain the highest values under the two multidimensional Gini indices and therefore exhibit the strongest concentration in their joint direct, indirect, and inferred severity-frequency distributions. These concentration patterns may help identify categories that warrant closer examination when mitigation priorities are determined.

26 pages, 6 figures