Cross-Entropy Games for Language Models: From Implicit Knowledge to General Capability Measures

Clément Hongler, Andrew Emil

Large Language Models (LLMs) define probability measures on text. By considering the implicit knowledge question of what it means for an LLM to know such a measure and what it enta…