2 papers
cs.CL2026
Tokenizer-Agnostic Engram Module
Jia Peng Lim, Hai Leong Chieu
Deepseek's Engram, a conditional memory module, was introduced to trade-off storage versus reasoning in large language models. However, the module relies on token-level -gram ha…
cs.CL2025
A Partition Cover Approach to Tokenization
Jia Peng Lim, Shawn Tan, Davin Choo +1
Tokenization is the process of encoding strings into tokens of a fixed vocabulary size, and is widely utilized in Natural Language Processing applications. The leading tokenization…