137 citations · 412 across the 61 of their papers we have counts for
62 papers
Asymmetric Feature Fusion for Image Retrieval
Hui Wu, Min Wang, Wengang Zhou +2
In asymmetric retrieval systems, models with different capacities are deployed on platforms with different computational and storage resources. Despite the great progress, existing…
UniDoc: A Universal Large Multimodal Model for Simultaneous Text Detection, Recognition, Spotting and Understanding
Hao Feng, Zijian Wang, Jingqun Tang +4
In the era of Large Language Models (LLMs), tremendous strides have been made in the field of multimodal understanding. However, existing advanced algorithms are limited to effecti…
Sign Language Translation with Iterative Prototype
Huijie Yao, Wengang Zhou, Hao Feng +3
This paper presents IP-SLT, a simple yet effective framework for sign language translation (SLT). Our IP-SLT adopts a recurrent structure and enhances the semantic representation (…
SimFIR: A Simple Framework for Fisheye Image Rectification with Self-supervised Representation Learning
Hao Feng, Wendi Wang, Jiajun Deng +3
In fisheye images, rich distinct distortion patterns are regularly distributed in the image plane. These distortion patterns are independent of the visual content and provide infor…
Text-Only Training for Visual Storytelling
Yuechen Wang, Wengang Zhou, Zhenbo Lu +1
Visual storytelling aims to generate a narrative based on a sequence of images, necessitating both vision-language alignment and coherent story generation. Most existing solutions…
Masked Motion Predictors are Strong 3D Action Representation Learners
Yunyao Mao, Jiajun Deng, Wengang Zhou +3
In 3D human action recognition, limited supervised data makes it challenging to fully tap into the modeling potential of powerful networks such as transformers. As a result, resear…
Cyclic-Bootstrap Labeling for Weakly Supervised Object Detection
Yufei Yin, Jiajun Deng, Wengang Zhou +2
Recent progress in weakly supervised object detection is featured by a combination of multiple instance detection networks (MIDN) and ordinal online refinement. However, with only…
Exploiting Spatial-Temporal Context for Interacting Hand Reconstruction on Monocular RGB Video
Weichao Zhao, Hezhen Hu, Wengang Zhou +2
Reconstructing interacting hands from monocular RGB data is a challenging task, as it involves many interfering factors, e.g. self- and mutual occlusion and similar textures. Previ…
AltFreezing for More General Video Face Forgery Detection
Zhendong Wang, Jianmin Bao, Wengang Zhou +2
Existing face forgery detection models try to discriminate fake images by detecting only spatial artifacts (e.g., generative artifacts, blending) or mainly temporal artifacts (e.g.…
Exploring Effective Mask Sampling Modeling for Neural Image Compression
Lin Liu, Mingming Zhao, Shanxin Yuan +5
Image compression aims to reduce the information redundancy in images. Most existing neural image compression methods rely on side information from hyperprior or context models to…
MA2CL:Masked Attentive Contrastive Learning for Multi-Agent Reinforcement Learning
Haolin Song, Mingxiao Feng, Wengang Zhou +1
Recent approaches have utilized self-supervised auxiliary tasks as representation learning to improve the performance and sample efficiency of vision-based reinforcement learning a…
Hybrid and Collaborative Passage Reranking
Zongmeng Zhang, Wengang Zhou, Jiaxin Shi +1
In passage retrieval system, the initial passage retrieval results may be unsatisfactory, which can be refined by a reranking scheme. Existing solutions to passage reranking focus…
SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understanding
Hezhen Hu, Weichao Zhao, Wengang Zhou +1
Hand gesture serves as a crucial role during the expression of sign language. Current deep learning based methods for sign language understanding (SLU) are prone to over-fitting du…
DocMAE: Document Image Rectification via Self-supervised Representation Learning
Shaokai Liu, Hao Feng, Wengang Zhou +3
Tremendous efforts have been made on document image rectification, but how to learn effective representation of such distorted images is still under-explored. In this paper, we pre…
Learning Transferable Pedestrian Representation from Multimodal Information Supervision
Liping Bao, Longhui Wei, Xiaoyu Qiu +3
Recent researches on unsupervised person re-identification~(reID) have demonstrated that pre-training on unlabeled person images achieves superior performance on downstream reID ta…
Deep Unrestricted Document Image Rectification
Hao Feng, Shaokai Liu, Jiajun Deng +2
In recent years, tremendous efforts have been made on document image rectification, but existing advanced algorithms are limited to processing restricted document images, i.e., the…
BEST: BERT Pre-Training for Sign Language Recognition with Coupling Tokenization
Weichao Zhao, Hezhen Hu, Wengang Zhou +2
In this work, we are dedicated to leveraging the BERT pre-training success and modeling the domain-specific statistics to fertilize the sign language recognition~(SLR) model. Consi…
HandNeRF: Neural Radiance Fields for Animatable Interacting Hands
Zhiyang Guo, Wengang Zhou, Min Wang +2
We propose a novel framework to reconstruct accurate appearance and geometry with neural radiance fields (NeRF) for interacting hands, enabling the rendering of photo-realistic ima…
NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation
Shengming Yin, Chenfei Wu, Huan Yang +13
In this paper, we propose NUWA-XL, a novel Diffusion over Diffusion architecture for eXtremely Long video generation. Most current work generates long videos segment by segment seq…
Human Pose as Compositional Tokens
Zigang Geng, Chunyu Wang, Yixuan Wei +3
Human pose is typically represented by a coordinate vector of body joints or their heatmap embeddings. While easy for data processing, unrealistic pose estimates are admitted due t…
DIRE for Diffusion-Generated Image Detection
Zhendong Wang, Jianmin Bao, Wengang Zhou +4
Diffusion models have shown remarkable success in visual synthesis, but have also raised concerns about potential abuse for malicious purposes. In this paper, we seek to build a de…
Focus on Your Target: A Dual Teacher-Student Framework for Domain-adaptive Semantic Segmentation
Xinyue Huo, Lingxi Xie, Wengang Zhou +2
We study unsupervised domain adaptation (UDA) for semantic segmentation. Currently, a popular UDA framework lies in self-training which endows the model with two-fold abilities: (i…
ROCO: A Roundabout Traffic Conflict Dataset
Depu Meng, Owen Sayer, Rusheng Zhang +3
Traffic conflicts have been studied by the transportation research community as a surrogate safety measure for decades. However, due to the rarity of traffic conflicts, collecting…
DIFFER: Decomposing Individual Reward for Fair Experience Replay in Multi-Agent Reinforcement Learning
Xunhan Hu, Jian Zhao, Wengang Zhou +2
Cooperative multi-agent reinforcement learning (MARL) is a challenging task, as agents must learn complex and diverse individual strategies from a shared team reward. However, exis…