PPLqa: An Unsupervised Information-Theoretic Quality Metric for Comparing Generative Large Language Models

Gerald Friedland, Xin Huang, Yueying Cui +3

We propose PPLqa, an easy to compute, language independent, information-theoretic metric to measure the quality of responses of generative Large Language Models (LLMs) in an unsupe…