6 papers
Parallel Rollout Approximation for Pixel-Space Autoregressive Image Generation
Jiayi Xu, Di He, Guolin Ke
Pixel-space continuous-token autoregressive (AR) generation directly models images as sequences of raw pixel patches, avoiding discrete tokenization or a separately pretrained toke…
One LR Doesn't Fit All: Heavy-Tail Guided Layerwise Learning Rates for LLMs
Di He, Songjun Tu, Keyu Wang +2
Learning rate configuration is a fundamental aspect of modern deep learning. The prevailing practice of applying a uniform learning rate across all layers overlooks the structural…
Quotient-Space Diffusion Models
Yixian Xu, Yusong Wang, Shengjie Luo +4
Diffusion-based generative models have reformed generative AI, and also enabled new capabilities in the science domain, e.g., fast generation of 3D structures of molecules. In such…
Lossless Anti-Distillation Sampling
Zibo Diao, Jingchu Gai, Xinyue Ai +3
Frontier commercial generative models face a growing threat from distillation, whereby a distiller harvests generated responses and trains a competing model of its own at drastical…
Luminark: Training-free, Probabilistically-Certified Watermarking for General Vision Generative Models
Jiayi Xu, Zhang Zhang, Yuanrui Zhang +4
In this paper, we introduce \emph{Luminark}, a training-free and probabilistically-certified watermarking method for general vision generative models. Our approach is built upon a…
Playing with Transformer at 30+ FPS via Next-Frame Diffusion
Xinle Cheng, Tianyu He, Jiayi Xu +3
Autoregressive video models offer distinct advantages over bidirectional diffusion models in creating interactive video content and supporting streaming applications with arbitrary…