EntropyMoE: Entropy-Aware Sparse Expert Routing for Tokenizer-Free LLMs

Bo Liu, Muxuab Yu, Yu Zhang +2

Recent byte-level large language models (LLMs) have made tokenizer-free modeling increasingly competitive by grouping bytes into dynamically sized patches. However, existing byte-p…