Shaofeng Zhang

works on

efficient training 1 large language models 1 residual stream expansion 1 sparse computation 1 transformer architectures 1

From the 1 of 12 linked papers with an AI index.

activity

20242026

collaborators

Showing cs.CLShow all

1 paper · 1 filter

cs.CL2026

NITP: Next Implicit Token Prediction for LLM Pre-training

Xiangdong Zhang, Debing Zhang, Shaofeng Zhang +3

Standard next-token prediction (NTP) supervises language models solely through discrete labels in the output logit space. We argue that this sparse one-hot supervision leaves the l…