5 papers
Reusing Operational Evidence After Context Changes: A Conservative Bayesian Framework for Autonomous Vehicle Safety
Robab Aghazadeh Chakherlou, Siddartha Khastgir, Xingyu Zhao
Operational evidence, i.e., evidence of operation without failure is an important component of confidence in the safety or reliability of a system in service, but it is costly to c…
Modeling Memory-Dependent Reliability of LLMs: A Hidden Markov Model
Robab Aghazadeh Chakherlou, Siddartha Khastgir, Peter Popov +1
Reliability assessment of large language models (LLMs) seeks to estimate the probability that a model produces correct responses under a specified operational profile. Conventional…
Uncertainty-Aware Measurement of Scenario Suite Representativeness for Autonomous Systems
Robab Aghazadeh Chakherlou, Siddartha Khastgir, Xingyu Zhao +2
Assuring the trustworthiness and safety of AI systems, e.g., autonomous vehicles (AV), depends critically on the data-related safety properties, e.g., representativeness, completen…
A Hierarchical Imprecise Probability Approach to Reliability Assessment of Large Language Models
Robab Aghazadeh-Chakherlou, Qing Guo, Siddartha Khastgir +3
Large Language Models (LLMs) are increasingly deployed across diverse domains, raising the need for rigorous reliability assessment methods. Existing benchmark-based evaluations pr…
A Scalable Framework for Safety Assurance of Self-Driving Vehicles based on Assurance 2.0
Shufeng Chen, Mariat James Elizebeth, Robab Aghazadeh Chakherlou +4
Assurance 2.0 is a modern framework developed to address the assurance challenges of increasingly complex, adaptive, and autonomous systems. Building on the traditional Claims-Argu…