349 citations · 1.4k across the 30 of their papers we have counts for
30 papers
StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generation
Yupeng Zhou, Daquan Zhou, Ming-Ming Cheng +2
For recent diffusion-based generative models, maintaining consistent content across a series of generated images, especially those containing subjects and complex details, presents…
Magic-Me: Identity-Specific Video Customized Diffusion
Ze Ma, Daquan Zhou, Chun-Hsiao Yeh +6
Creating content with specified identities (ID) has attracted significant interest in the field of generative models. In the field of text-to-image generation (T2I), subject-driven…
Sora Generates Videos with Stunning Geometrical Consistency
Xuanyi Li, Daquan Zhou, Chenxu Zhang +3
The recently developed Sora model [1] has exhibited remarkable capabilities in video generation, sparking intense discussions regarding its ability to simulate real-world phenomena…
ChatAnything: Facetime Chat with LLM-Enhanced Personas
Yilin Zhao, Xinbin Yuan, Shanghua Gao +4
In this technical report, we target generating anthropomorphized personas for LLM-based characters in an online manner, including visual appearance, personality and tones, with onl…
MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaboration
Lin Xu, Zhiyuan Hu, Daquan Zhou +5
Large Language Models (LLMs) have significantly advanced natural language processing, demonstrating exceptional reasoning, tool usage, and memory capabilities. As their application…
EPIM: Efficient Processing-In-Memory Accelerators based on Epitome
Chenyu Wang, Zhen Dong, Daquan Zhou +4
The utilization of large-scale neural networks on Processing-In-Memory (PIM) accelerators encounters challenges due to constrained on-chip memory capacity. To tackle this issue, cu…
Low-Resolution Self-Attention for Semantic Segmentation
Yu-Huan Wu, Shi-Chen Zhang, Yun Liu +6
Semantic segmentation tasks naturally require high-resolution information for pixel-wise segmentation and global context information for class prediction. While existing vision tra…
MaskDiffusion: Boosting Text-to-Image Consistency with Conditional Mask
Yupeng Zhou, Daquan Zhou, Zuo-Liang Zhu +3
Recent advancements in diffusion models have showcased their impressive capacity to generate visually striking images. Nevertheless, ensuring a close match between the generated im…
Dataset Quantization
Daquan Zhou, Kai Wang, Jianyang Gu +5
State-of-the-art deep neural networks are trained with large amounts (millions or even billions) of data. The expensive computation and memory costs make it difficult to train them…
BuboGPT: Enabling Visual Grounding in Multi-Modal LLMs
Yang Zhao, Zhijie Lin, Daquan Zhou +3
LLMs have demonstrated remarkable abilities at interacting with humans through language, especially with the usage of instruction-following data. Recent advancements in LLMs, such…
DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuning
Enze Xie, Lewei Yao, Han Shi +5
Diffusion models have proven to be highly effective in generating high-quality images. However, adapting large pre-trained diffusion models to new domains remains an open challenge…
InfoBatch: Lossless Training Speed Up by Unbiased Dynamic Data Pruning
Ziheng Qin, Kai Wang, Zangwei Zheng +8
Data pruning aims to obtain lossless performances with less overall cost. A common approach is to filter out samples that make less contribution to the training. This could lead to…
Diffusion Probabilistic Model Made Slim
Xingyi Yang, Daquan Zhou, Jiashi Feng +1
Despite the recent visually-pleasing results achieved, the massive computational cost has been a long-standing flaw for diffusion probabilistic models (DPMs), which, in turn, great…
Expanding Small-Scale Datasets with Guided Imagination
Yifan Zhang, Daquan Zhou, Bryan Hooi +2
The power of DNNs relies heavily on the quantity and quality of training data. However, collecting and annotating data on a large scale is often expensive and time-consuming. To ad…
MagicVideo: Efficient Video Generation With Latent Diffusion Models
Daquan Zhou, Weimin Wang, Hanshu Yan +3
We present an efficient text-to-video generation framework based on latent diffusion models, termed MagicVideo. MagicVideo can generate smooth video clips that are concordant with…
MagicMix: Semantic Mixing with Diffusion Models
Jun Hao Liew, Hanshu Yan, Daquan Zhou +1
Have you ever imagined what a corgi-alike coffee machine or a tiger-alike rabbit would look like? In this work, we attempt to answer these questions by exploring a new task called…
Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning
Dongze Lian, Daquan Zhou, Jiashi Feng +1
Existing fine-tuning methods either tune all parameters of the pre-trained model (full fine-tuning), which is not efficient, or only tune the last linear layer (linear probing), wh…
Sharpness-Aware Training for Free
Jiawei Du, Daquan Zhou, Jiashi Feng +2
Modern deep neural networks (DNNs) have achieved state-of-the-art performances but are typically over-parameterized. The over-parameterization may result in undesirably large gener…
MBEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representation
Enze Xie, Zhiding Yu, Daquan Zhou +5
In this paper, we propose MBEV, a unified framework that jointly performs 3D object detection and map segmentation in the Birds Eye View~(BEV) space with multi-camera image inp…
Understanding The Robustness in Vision Transformers
Daquan Zhou, Zhiding Yu, Enze Xie +4
Recent studies show that Vision Transformers(ViTs) exhibit strong robustness against various corruptions. Although this property is partly attributed to the self-attention mechanis…
Shunted Self-Attention via Multi-Scale Token Aggregation
Sucheng Ren, Daquan Zhou, Shengfeng He +2
Recent Vision Transformer~(ViT) models have demonstrated encouraging results across various computer vision tasks, thanks to their competence in modeling long-range dependencies of…
Refiner: Refining Self-attention for Vision Transformers
Daquan Zhou, Yujun Shi, Bingyi Kang +6
Vision Transformers (ViTs) have shown competitive accuracy in image classification tasks compared with CNNs. Yet, they generally require much more data for model pre-training. Most…
DeepViT: Towards Deeper Vision Transformer
Daquan Zhou, Bingyi Kang, Xiaojie Jin +5
Vision transformers (ViTs) have been successfully applied in image classification tasks recently. In this paper, we show that, unlike convolution neural networks (CNNs)that can be…
All Tokens Matter: Token Labeling for Training Better Vision Transformers
Zihang Jiang, Qibin Hou, Li Yuan +5
In this paper, we present token labeling -- a new training objective for training high-performance vision transformers (ViTs). Different from the standard training objective of ViT…