JTok: On Token Embedding as another Axis of Scaling Law via Joint Token Self-modulation

Yebin Yang, Huaijin Wu, Fu Guo +5

LLMs have traditionally scaled along dense dimensions, where performance is coupled with near-linear increases in computational cost. While MoE decouples capacity from compute, it…