Evidence of Phase Transitions in Small Transformer-Based Language Models

Noah Hong, Tao Hong

Phase transitions have been proposed as the origin of emergent abilities in large language models (LLMs), where new capabilities appear abruptly once models surpass critical thresh…