Contextual Morphogenesis in Large Language Models: A Novel Approach to Self-Organizing Token Representations

Alistair Dombrowski, Beatrix Engelhardt, Dimitri Fairbrother +1

Token representations influence the efficiency and adaptability of language models, yet conventional tokenization strategies impose rigid segmentation boundaries that do not adjust…