116 citations · 199 across the 17 of their papers we have counts for
17 papers
PaRa: Personalizing Text-to-Image Diffusion via Parameter Rank Reduction
Shangyu Chen, Zizheng Pan, Jianfei Cai +1
Personalizing a large-scale pretrained Text-to-Image (T2I) diffusion model is challenging as it typically struggles to make an appropriate trade-off between its training data distr…
MiniCache: KV Cache Compression in Depth Dimension for Large Language Models
Akide Liu, Jing Liu, Zizheng Pan +3
A critical approach for efficiently deploying computationally demanding large language models (LLMs) is Key-Value (KV) caching. The KV cache stores key-value states of previously g…
T-Stitch: Accelerating Sampling in Pre-Trained Diffusion Models with Trajectory Stitching
Zizheng Pan, Bohan Zhuang, De-An Huang +5
Sampling from diffusion probabilistic models (DPMs) is often expensive for high-quality image generation and typically requires many steps with a large model. In this paper, we int…
Efficient Stitchable Task Adaptation
Haoyu He, Zizheng Pan, Jing Liu +2
The paradigm of pre-training and fine-tuning has laid the foundation for deploying deep learning models. However, most fine-tuning methods are designed to meet a specific resource…
Stitched ViTs are Flexible Vision Backbones
Zizheng Pan, Jing Liu, Haoyu He +2
Large pretrained plain vision Transformers (ViTs) have been the workhorse for many downstream tasks. However, existing works utilizing off-the-shelf ViTs are inefficient in terms o…
Stitchable Neural Networks
Zizheng Pan, Jianfei Cai, Bohan Zhuang
The public model zoo containing enormous powerful pretrained model families (e.g., ResNet/DeiT) has reached an unprecedented scope than ever, which significantly contributes to the…
A Survey on Efficient Training of Transformers
Bohan Zhuang, Jing Liu, Zizheng Pan +3
Recent advances in Transformers have come with a huge requirement on computing resources, highlighting the importance of developing efficient training techniques to make Transforme…
EcoFormer: Energy-Saving Attention with Linear Complexity
Jing Liu, Zizheng Pan, Haoyu He +2
Transformer is a transformative framework that models sequential data and has achieved remarkable performance on a wide range of tasks, but with high computational and energy cost.…
An Efficient Spatio-Temporal Pyramid Transformer for Action Detection
Yuetian Weng, Zizheng Pan, Mingfei Han +2
The task of action detection aims at deducing both the action category and localization of the start and end moment for each action instance in a long, untrimmed video. While visio…
Fast Vision Transformers with HiLo Attention
Zizheng Pan, Jianfei Cai, Bohan Zhuang
Vision Transformers (ViTs) have triggered the most recent and significant breakthroughs in computer vision. Their efficient designs are mostly guided by the indirect metric of comp…
Dynamic Focus-aware Positional Queries for Semantic Segmentation
Haoyu He, Jianfei Cai, Zizheng Pan +4
The DETR-like segmentors have underpinned the most recent breakthroughs in semantic segmentation, which end-to-end train a set of queries representing the class prototypes or targe…
Pruning Self-attentions into Convolutional Layers in Single Path
Haoyu He, Jianfei Cai, Jing Liu +4
Vision Transformers (ViTs) have achieved impressive performance over various computer vision tasks. However, modeling global correlations with multi-head self-attention (MSA) layer…
Mesa: A Memory-saving Training Framework for Transformers
Zizheng Pan, Peng Chen, Haoyu He +3
There has been an explosion of interest in designing high-performance Transformers. While Transformers have delivered significant performance improvements, training such networks i…
Less is More: Pay Less Attention in Vision Transformers
Zizheng Pan, Bohan Zhuang, Haoyu He +2
Transformers have become one of the dominant architectures in deep learning, particularly as a powerful alternative to convolutional neural networks (CNNs) in computer vision. Howe…
The Road to Know-Where: An Object-and-Room Informed Sequential BERT for Indoor Vision-Language Navigation
Yuankai Qi, Zizheng Pan, Yicong Hong +3
Vision-and-Language Navigation (VLN) requires an agent to find a path to a remote location on the basis of natural-language instructions and a set of photo-realistic panoramas. Mos…
Scalable Vision Transformers with Hierarchical Pooling
Zizheng Pan, Bohan Zhuang, Jing Liu +2
The recently proposed Visual image Transformers (ViT) with pure attention have achieved promising performance on image recognition tasks, such as image classification. However, the…
Object-and-Action Aware Model for Visual Language Navigation
Yuankai Qi, Zizheng Pan, Shengping Zhang +2
Vision-and-Language Navigation (VLN) is unique in that it requires turning relatively general natural-language instructions into robot agent actions, on the basis of the visible en…