State space models, emergence, and ergodicity: How many parameters are needed for stable predictions?

Ingvar Ziemann, Nikolai Matni, George J. Pappas

How many parameters are required for a model to execute a given task? It has been argued that large language models, pre-trained via self-supervised learning, exhibit emergent capa…