263 citations · 495 across the 16 of their papers we have counts for
17 papers
Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generation
Zhiheng Liu, Weiming Ren, Xiaoke Huang +12
Unified multimodal models typically rely on pretrained vision encoders and use separate visual representations for understanding and generation, creating misalignment between the t…
PixelFlow: Pixel-Space Generative Models with Flow
Shoufa Chen, Chongjian Ge, Shilong Zhang +2
We present PixelFlow, a family of image generation models that operate directly in the raw pixel space, in contrast to the predominant latent-space models. This approach simplifies…
Prompt-A-Video: Prompt Your Video Diffusion Model via Preference-Aligned LLM
Yatai Ji, Jiacheng Zhang, Jie Wu +9
Text-to-video models have made remarkable advancements through optimization on high-quality text-video pairs, where the textual prompts play a pivotal role in determining quality o…
ControlAR: Controllable Image Generation with Autoregressive Models
Zongming Li, Tianheng Cheng, Shoufa Chen +6
Autoregressive (AR) models have reformulated image generation as next-token prediction, demonstrating remarkable potential and emerging as strong competitors to diffusion models. H…
Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation
Peize Sun, Yi Jiang, Shoufa Chen +4
We introduce LlamaGen, a new family of image generation models that apply original ``next-token prediction'' paradigm of large language models to visual generation domain. It is an…
RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesis
Yao Mu, Junting Chen, Qinglong Zhang +16
Robotic behavior synthesis, the problem of understanding multimodal inputs and generating precise physical control for robots, is an important part of Embodied AI. Despite successe…
Enhancing Your Trained DETRs with Box Refinement
Yiqun Chen, Qiang Chen, Peize Sun +3
We present a conceptually simple, efficient, and general framework for localization problems in DETR-like models. We add plugins to well-trained models instead of inefficiently des…
GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest
Shilong Zhang, Peize Sun, Shoufa Chen +6
Visual instruction tuning large language model(LLM) on image-text pairs has achieved general-purpose vision-language abilities. However, the lack of region-text pairs limits their…
InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Language
Zhaoyang Liu, Yinan He, Wenhai Wang +17
We present an interactive visual framework named InternGPT, or iGPT for short. The framework integrates chatbots that have planning and reasoning capabilities, such as ChatGPT, wit…
Going Denser with Open-Vocabulary Part Segmentation
Peize Sun, Shoufa Chen, Chenchen Zhu +4
Object detection has been expanded from a limited number of categories to open vocabulary. Moving forward, a complete intelligent vision system requires understanding more fine-gra…
Soft Neighbors are Positive Supporters in Contrastive Visual Representation Learning
Chongjian Ge, Jiangliu Wang, Zhan Tong +3
Contrastive learning methods train visual encoders by comparing views from one instance to others. Typically, the views created from one instance are set as positive, while views f…
DiffusionDet: Diffusion Model for Object Detection
Shoufa Chen, Peize Sun, Yibing Song +1
We propose DiffusionDet, a new framework that formulates object detection as a denoising diffusion process from noisy boxes to object boxes. During the training stage, object boxes…
CtrlFormer: Learning Transferable State Representation for Visual Control via Transformer
Yao Mu, Shoufa Chen, Mingyu Ding +3
Transformer has achieved great successes in learning vision and language representation, which is general across various downstream tasks. In visual control, learning transferable…
AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition
Shoufa Chen, Chongjian Ge, Zhan Tong +4
Pretraining Vision Transformers (ViTs) has achieved great success in visual recognition. A following scenario is to adapt a ViT to various image and video recognition tasks. The ad…
Towards High-Quality Temporal Action Detection with Sparse Proposals
Jiannan Wu, Peize Sun, Shoufa Chen +4
Temporal Action Detection (TAD) is an essential and challenging topic in video understanding, aiming to localize the temporal segments containing human action instances and predict…
CycleMLP: A MLP-like Architecture for Dense Prediction
Shoufa Chen, Enze Xie, Chongjian Ge +3
This paper presents a simple MLP-like architecture, CycleMLP, which is a versatile backbone for visual recognition and dense predictions. As compared to modern MLP architectures, e…
Efficient Differentiable Neural Architecture Search with Meta Kernels
Shoufa Chen, Yunpeng Chen, Shuicheng Yan +1
The searching procedure of neural architecture search (NAS) is notoriously time consuming and cost prohibitive.To make the search space continuous, most existing gradient-based NAS…