14 papers
Large Language Models are Universal Reasoners for Visual Generation
Sucheng Ren, Chen Chen, Zhenbang Wang +5
Text-to-image generation has advanced rapidly with diffusion models, progressing from CLIP and T5 conditioning to unified systems where a single LLM backbone handles both visual un…
Frequency-Aware Flow Matching for High-Quality Image Generation
Sucheng Ren, Qihang Yu, Ju He +3
Flow matching models have emerged as a powerful framework for realistic image generation by learning to reverse a corruption process that progressively adds Gaussian noise. However…
ViT-5: Vision Transformers for The Mid-2020s
Feng Wang, Sucheng Ren, Tiezheng Zhang +4
This work presents a systematic investigation into modernizing Vision Transformer backbones by leveraging architectural advancements from the past five years. While preserving the…
WorldEdit: Towards Open-World Image Editing with a Knowledge-Informed Benchmark
Wang Lin, Feng Wang, Majun Zhang +7
Recent advances in image editing models have demonstrated remarkable capabilities in executing explicit instructions, such as attribute manipulation, style transfer, and pose synth…
CoPE: Clipped RoPE as A Scalable Free Lunch for Long Context LLMs
Haoran Li, Sucheng Ren, Alan Yuille +1
Rotary Positional Embedding (RoPE) is a key component of context scaling in Large Language Models (LLMs). While various methods have been proposed to adapt RoPE to longer contexts,…
Spiral RoPE: Rotate Your Rotary Positional Embeddings in the 2D Plane
Haoyu Liu, Sucheng Ren, Tingyu Zhu +5
Rotary Position Embedding (RoPE) is the de facto positional encoding in large language models due to its ability to encode relative positions and support length extrapolation. When…