3 papers
cs.CL2026
Language Models Without a Trainable Input Embedding Table: Learning from Fixed Minimal Binary Token Codes
A. Bochkov
Trainable input embedding tables are a standard component of modern language models. We ask whether they are actually necessary at the input interface. For a vocabulary of size …
cs.LG2026
Growing Transformers: Modular Composition and Layer-wise Expansion on a Frozen Substrate
A. Bochkov
We study a constrained training regime for decoder-only Transformers in which the token interface is fixed, previously trained dense blocks are not reopened, and the active trainab…
cs.CL2025
Emergent Semantics Beyond Token Embeddings: Transformer LMs with Frozen Visual Unicode Representations
A. Bochkov
Understanding the locus of semantic representation in large language models (LLMs) is crucial for interpretability and architectural innovation. The dominant paradigm posits that t…