263 citations · 719 across the 46 of their papers we have counts for
17 papers · 1 filter
Policy Adaptation from Foundation Model Feedback
Yuying Ge, Annabella Macaluso, Li Erran Li +2
Recent progress on vision-language foundation models have brought significant advancement to building general-purpose robots. By using the pre-trained models to encode the scene an…
Prototypical context-aware dynamics generalization for high-dimensional model-based reinforcement learning
Junjie Wang, Yao Mu, Dong Li +6
The latent world model provides a promising way to learn policies in a compact latent space for tasks with high-dimensional observations, however, its generalization across diverse…
Large-batch Optimization for Dense Visual Predictions
Zeyue Xue, Jianming Liang, Guanglu Song +4
Training a large-scale deep neural network in a large-scale dataset is challenging and time-consuming. The recent breakthrough of large-batch optimization is a promising way to tac…
Enhance Sample Efficiency and Robustness of End-to-end Urban Autonomous Driving via Semantic Masked World Model
Zeyu Gao, Yao Mu, Chen Chen +4
End-to-end autonomous driving provides a feasible way to automatically maximize overall driving system performance by directly mapping the raw pixels from a front-facing camera to…
Rethinking Resolution in the Context of Efficient Video Recognition
Chuofan Ma, Qiushan Guo, Yi Jiang +3
In this paper, we empirically study how to make the most of low-resolution frames for efficient video recognition. Existing methods mainly focus on developing compact networks or a…
Learning Transferable Spatiotemporal Representations from Natural Script Knowledge
Ziyun Zeng, Yuying Ge, Xihui Liu +4
Pre-training on large-scale video data has become a common recipe for learning transferable spatiotemporal representations in recent years. Despite some progress, existing methods…
Real-time End-to-End Video Text Spotter with Contrastive Representation Learning
Wejia Wu, Zhuang Li, Jiahong Li +5
Video text spotting(VTS) is the task that requires simultaneously detecting, tracking and recognizing text in the video. Existing video text spotting methods typically develop soph…
Not All Models Are Equal: Predicting Model Transferability in a Self-challenging Fisher Space
Wenqi Shao, Xun Zhao, Yixiao Ge +5
This paper addresses an important problem of ranking the pre-trained deep neural networks and screening the most transferable ones for downstream tasks. It is challenging because t…
Exploiting Context Information for Generic Event Boundary Captioning
Jinrui Zhang, Teng Wang, Feng Zheng +2
Generic Event Boundary Captioning (GEBC) aims to generate three sentences describing the status change for a given time boundary. Previous methods only process the information of a…
VLMixer: Unpaired Vision-Language Pre-training via Cross-Modal CutMix
Teng Wang, Wenhao Jiang, Zhichao Lu +4
Existing vision-language pre-training (VLP) methods primarily rely on paired image-text datasets, which are either annotated by enormous human labors, or crawled from the internet…
CtrlFormer: Learning Transferable State Representation for Visual Control via Transformer
Yao Mu, Shoufa Chen, Mingyu Ding +3
Transformer has achieved great successes in learning vision and language representation, which is general across various downstream tasks. In visual control, learning transferable…
AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition
Shoufa Chen, Chongjian Ge, Zhan Tong +4
Pretraining Vision Transformers (ViTs) has achieved great success in visual recognition. A following scenario is to adapt a ViT to various image and video recognition tasks. The ad…
MILES: Visual BERT Pre-training with Injected Language Semantics for Video-text Retrieval
Yuying Ge, Yixiao Ge, Xihui Liu +5
Dominant pre-training work for video-text retrieval mainly adopt the "dual-encoder" architectures to enable efficient retrieval, where two separate encoders are used to contrast gl…
Semantic-Aware Pretraining for Dense Video Captioning
Teng Wang, Zhu Liu, Feng Zheng +3
This report describes the details of our approach for the event dense-captioning task in ActivityNet Challenge 2021. We present a semantic-aware pretraining method for dense video…
Compression of Generative Pre-trained Language Models via Quantization
Chaofan Tao, Lu Hou, Wei Zhang +5
The increasing size of generative Pre-trained Language Models (PLMs) has greatly increased the demand for model compression. Despite various methods to compress BERT or its variant…
End-to-End Video Text Spotting with Transformer
Weijia Wu, Yuanqiang Cai, Chunhua Shen +4
Recent video text spotting methods usually require the three-staged pipeline, i.e., detecting text in individual images, recognizing localized text, tracking text streams with post…
Bridging Video-text Retrieval with Multiple Choice Questions
Yuying Ge, Yixiao Ge, Xihui Liu +4
Pre-training a model to learn transferable video-text representation for retrieval has attracted a lot of attention in recent years. Previous dominant works mainly adopt two separa…