Forget BIT, It is All about TOKEN: Towards Semantic Information Theory for LLMs

Bo Bai

Despite the empirical successes of Large Language Models (LLMs), the prevailing paradigm is heuristic and experiment-driven, tethered to massive compute and data, while a first-pri…