3 papers
cs.CL2026
Language Models Without a Trainable Input Embedding Table: Learning from Fixed Minimal Binary Token Codes
A. Bochkov
We study whether a decoder-only language model requires an independently trainable input vector for every token. For a vocabulary of size , an injective fixed-length binary iden…
cs.LG2025
Growing Transformers: Modular Composition and Layer-wise Expansion on a Frozen Substrate
A. Bochkov
We study a constrained training regime for decoder-only Transformers in which the token interface is fixed, previously trained dense blocks are not reopened, and the active trainab…
cs.CL2025
Emergent Semantics Beyond Token Embeddings: Transformer LMs with Frozen Visual Unicode Representations
A. Bochkov
Understanding the locus of semantic representation in large language models (LLMs) is crucial for interpretability and architectural innovation. The dominant paradigm posits that t…