35 citations · 321 across the 48 of their papers we have counts for
50 papers
FLatten Transformer: Vision Transformer using Focused Linear Attention
Dongchen Han, Xuran Pan, Yizeng Han +2
The quadratic computation complexity of self-attention has been a persistent challenge when applying Transformer models to vision tasks. Linear attention, on the other hand, offers…
Computation-efficient Deep Learning for Computer Vision: A Survey
Yulin Wang, Yizeng Han, Chaofei Wang +3
Over the past decade, deep learning models have exhibited considerable advancements, reaching or even exceeding human-level performance in a range of visual perception tasks. This…
Learning Specialized Activation Functions for Physics-informed Neural Networks
Honghui Wang, Lu Lu, Shiji Song +1
Physics-informed neural networks (PINNs) are known to suffer from optimization difficulty. In this work, we reveal the connection between the optimization difficulty of PINNs and a…
Latency-aware Unified Dynamic Networks for Efficient Image Recognition
Yizeng Han, Zeyu Liu, Zhihang Yuan +4
Dynamic computation has emerged as a promising avenue to enhance the inference efficiency of deep networks. It allows selective activation of computational units, leading to a redu…
ExpeL: LLM Agents Are Experiential Learners
Andrew Zhao, Daniel Huang, Quentin Xu +3
The recent surge in research interest in applying large language models (LLMs) to decision-making tasks has flourished by leveraging the extensive world knowledge embedded in LLMs.…
Boosting Offline Reinforcement Learning with Action Preference Query
Qisen Yang, Shenzhi Wang, Matthieu Gaetan Lin +2
Training practical agents usually involve offline and online reinforcement learning (RL) to balance the policy's performance and interaction costs. In particular, online fine-tunin…
Dynamic Perceiver for Efficient Visual Recognition
Yizeng Han, Dongchen Han, Zeyu Liu +7
Early exiting has become a promising approach to improving the inference efficiency of deep networks. By structuring models with multiple classifiers (exits), predictions for ``eas…
ADDP: Learning General Representations for Image Recognition and Generation with Alternating Denoising Diffusion Process
Changyao Tian, Chenxin Tao, Jifeng Dai +7
Image recognition and generation have long been developed independently of each other. With the recent trend towards general-purpose representation learning, the development of gen…
Decoupled Prioritized Resampling for Offline RL
Yang Yue, Bingyi Kang, Xiao Ma +4
Offline reinforcement learning (RL) is challenged by the distributional shift problem. To address this problem, existing works mainly focus on designing sophisticated policy constr…
Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory
Xizhou Zhu, Yuntao Chen, Hao Tian +10
The captivating realm of Minecraft has attracted substantial research interest in recent years, serving as a rich platform for developing intelligent agents capable of functioning…
Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models
Xingqian Xu, Jiayi Guo, Zhangyang Wang +3
Text-to-image (T2I) research has grown explosively in the past year, owing to the large-scale pre-trained diffusion models and many emerging personalization and editing approaches.…
Slide-Transformer: Hierarchical Vision Transformer with Local Self-Attention
Xuran Pan, Tianzhu Ye, Zhuofan Xia +2
Self-attention mechanism has been a key factor in the recent progress of Vision Transformer (ViT), which enables adaptive feature extraction from global contexts. However, existing…
Zero-shot Generative Model Adaptation via Image-specific Prompt Learning
Jiayi Guo, Chaofei Wang, You Wu +6
Recently, CLIP-guided image synthesis has shown appealing performance on adapting a pre-trained source-domain generator to an unseen target domain. It does not require any target-d…
Adaptive Rotated Convolution for Rotated Object Detection
Yifan Pu, Yiru Wang, Zhuofan Xia +6
Rotated object detection aims to identify and locate objects in images with arbitrary orientation. In this scenario, the oriented directions of objects vary considerably across dif…
Joint Representation Learning for Text and 3D Point Cloud
Rui Huang, Xuran Pan, Henry Zheng +4
Recent advancements in vision-language pre-training (e.g. CLIP) have shown that vision models can benefit from language supervision. While many models using language modality have…
Convolution-enhanced Evolving Attention Networks
Yujing Wang, Yaming Yang, Zhuo Li +7
Attention-based neural networks, such as Transformers, have become ubiquitous in numerous applications, including computer vision, natural language processing, and time-series anal…
Deep Incubation: Training Large Models by Divide-and-Conquering
Zanlin Ni, Yulin Wang, Jiangwei Yu +3
Recent years have witnessed a remarkable success of large deep learning models. However, training these models is challenging due to high computational costs, painfully slow conver…
Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information
Weijie Su, Xizhou Zhu, Chenxin Tao +7
To effectively exploit the potential of large-scale models, various pre-training strategies supported by massive data from different sources are proposed, including supervised pre-…
BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision
Chenyu Yang, Yuntao Chen, Hao Tian +9
We present a novel bird's-eye-view (BEV) detector with perspective supervision, which converges faster and better suits modern image backbones. Existing state-of-the-art BEV detect…
EfficientTrain: Exploring Generalized Curriculum Learning for Training Visual Backbones
Yulin Wang, Yang Yue, Rui Lu +4
The superior performance of modern deep networks usually comes with a costly training procedure. This paper presents a new curriculum learning approach for the efficient training o…
Cross-Modal Adapter for Vision-Language Retrieval
Haojun Jiang, Jianke Zhang, Rui Huang +4
Vision-language retrieval is an important multi-modal learning topic, where the goal is to retrieve the most relevant visual candidate for a given text query. Recently, pre-trained…
Boosting Offline Reinforcement Learning via Data Rebalancing
Yang Yue, Bingyi Kang, Xiao Ma +3
Offline reinforcement learning (RL) is challenged by the distributional shift between learning policies and datasets. To address this problem, existing works mainly focus on design…
Contrastive Language-Image Pre-Training with Knowledge Graphs
Xuran Pan, Tianzhu Ye, Dongchen Han +2
Recent years have witnessed the fast development of large-scale pre-training frameworks that can extract multi-modal representations in a unified form and achieve promising perform…
A Mixture of Surprises for Unsupervised Reinforcement Learning
Andrew Zhao, Matthieu Gaetan Lin, Yangguang Li +2
Unsupervised reinforcement learning aims at learning a generalist policy in a reward-free manner for fast adaptation to downstream tasks. Most of the existing methods propose to pr…