1 citations · 3 across the 6 of their papers we have counts for
4 papers · 1 filter
Aperture: Merge-Consistent Rotary States for Compressed Tokens
Yuhao Du, Shunian Chen
Token compression combines content from several positions, yet rotary position embeddings usually assign the merged token one coordinate. We ask what positional information must su…
The Price of Token Boundaries: Compression Certificates and Prediction
Yuhao Du, Shunian Chen
Pre-tokenisation restricts which text fragments can become prediction units, but its compression cost is obscured when tokenisers are compared only under the same boundaries. We me…
Refinement Symmetry in Multimodal Transformers
Yuhao Du, Shunian Chen
Attention weights depend on token counts, which change with the representation of a signal. We study refinement symmetry: splitting a representation while preserving content, posit…
World Models with Predictable Long-Horizon Marginals
Yuhao Du, Shunian Chen
Accurate one-step predictions do not ensure that a world model's rollouts retain the data distribution. We make the model's decoded stationary law explicit by learning a decoder of…