Modeling Memory-Dependent Reliability of LLMs: A Hidden Markov Model

Robab Aghazadeh Chakherlou, Siddartha Khastgir, Peter Popov +1

Reliability assessment of large language models (LLMs) seeks to estimate the probability that a model produces correct responses under a specified operational profile. Conventional…