Next Token Perception Score: Analytical Assessment of your LLM Perception Skills

Yu-Ang Cheng, Leyang Hu, Hai Huang +1

Autoregressive pretraining has become the de facto paradigm for learning general-purpose representations in large language models (LLMs). However, linear probe performance across d…