21 papers
Symmetric Divergence and Normalized Similarity: A Unified Topological Framework for Representation Analysis
Yan Wang, Tianyang Hu
Topological Data Analysis (TDA) offers a principled, intrinsic lens for comparing neural representations. However, existing paired topological divergences (e.g., RTD) are limited b…
Autoregressive Visual Generation Needs a Prologue
Bowen Zheng, Weijian Luo, Guang Yang +2
In this work, we propose Prologue, an approach to bridging the reconstruction-generation gap in autoregressive (AR) image generation. Instead of modifying visual tokens to satisfy…
The Structural Origin of Attention Sink: Variance Discrepancy, Super Neurons, and Dimension Disparity
Siquan Li, Kaiqi Jiang, Jiacheng Sun +1
Despite the prevalence of the attention sink phenomenon in Large Language Models (LLMs), where initial tokens disproportionately monopolize attention scores, its structural origins…
Taming the Entropy Cliff: Variable Codebook Size Quantization for Autoregressive Visual Generation
Bowen Zheng, Weijian Luo, Guang Yang +2
Most discrete visual tokenizers rely on a default design: every position in the sequence shares the same codebook. Researchers try to scale the codebook size to get better reco…
Learning Discrete Autoregressive Priors with Wasserstein Gradient Flow
Bowen Zheng, Yihong Luo, Tianyang Hu
Discrete image tokenizers are commonly trained in two stages: first for reconstruction, and then with a prior model fitted to the frozen token sequences. This decoupling leaves the…
PrefixMemory-Tuning: Modernizing Prefix-Tuning by Decoupling the Prefix from Attention
Haonan Wang, Brian Chen, Siquan Li +4
Parameter-Efficient Fine-Tuning (PEFT) methods have become crucial for rapidly adapting large language models (LLMs) to downstream tasks. Prefix-Tuning, an early and effective PEFT…