The Dynamics of Continuous Mixture Collapse in Language Models

Ali Backour

LLMs latent-state reasoning methods replace discrete intermediate tokens with continuous states, such as weighted mixtures of token embeddings, to retain multiple possible reasonin…