4 papers
A Conditional No-Hair Theorem for Transformers: How Language Models Forget Their Prompt
Shuyang Xiang
We ask how much an autoregressive transformer retains of its prompt after generating tokens. Consider two different prompts followed by the same continuation text, so that any…
Paragraph Boundaries Are Not White Space:Compression Depth as the Signature of Hierarchical Structure
Shuyang Xiang
Standard positional encodings treat position as a one-dimensional reading-order coordinate, but reading order alone does not determine hierarchical textual structure. We use a hier…
One Image, No Tokens: A Controlled Study of Glyph-Based Chinese Language Modeling
Shuyang Xiang, Hao Guan
Modern language models generally represent text as sequences of discrete token embeddings, an assumption deeply rooted in current practice but rarely questioned. We challenge this…
Hot-Start Chinese Language Modeling:Visual Glyphs Accelerate Sample-Efficient Learning
Shuyang Xiang, Hao Guan
In this work, we study whether rendering Chinese characters as visual glyph images, rather than discrete token IDs as mainstream LLMs do, providing an inductive bias for character-…