273 citations · 1.4k across the 86 of their papers we have counts for
29 papers · 1 filter
Amodal Ground Truth and Completion in the Wild
Guanqi Zhan, Chuanxia Zheng, Weidi Xie +1
This paper studies amodal image segmentation: predicting entire object segmentation masks including both visible and invisible (occluded) parts. In previous work, the amodal segmen…
Appearance-Based Refinement for Object-Centric Motion Segmentation
Junyu Xie, Weidi Xie, Andrew Zisserman
The goal of this paper is to discover, segment, and track independently moving objects in complex visual scenes. Previous approaches have explored the use of optical flow for motio…
Self-supervised Object-Centric Learning for Videos
Görkay Aydemir, Weidi Xie, Fatma Güney
Unsupervised multi-object segmentation has shown impressive results on images by utilizing powerful semantics learned from self-supervised pretraining. An additional modality such…
AutoAD II: The Sequel -- Who, When, and What in Movie Audio Description
Tengda Han, Max Bain, Arsha Nagrani +3
Audio Description (AD) is the task of generating descriptions of visual content, at suitable time intervals, for the benefit of visually impaired audiences. For movies, this presen…
A General Protocol to Probe Large Vision Models for 3D Physical Understanding
Guanqi Zhan, Chuanxia Zheng, Weidi Xie +1
Our objective in this paper is to probe large vision models to determine to what extent they 'understand' different physical properties of the 3D scene depicted in an image. To thi…
The Making and Breaking of Camouflage
Hala Lamdouar, Weidi Xie, Andrew Zisserman
Not all camouflages are equally effective, as even a partially visible contour or a slight color difference can make the animal stand out and break its camouflage. In this paper, w…
Diagnosing Human-object Interaction Detectors
Fangrui Zhu, Yiming Xie, Weidi Xie +1
We have witnessed significant progress in human-object interaction (HOI) detection. The reliance on mAP (mean Average Precision) scores as a summary metric, however, does not provi…
Joint-Relation Transformer for Multi-Person Motion Prediction
Qingyao Xu, Weibo Mao, Jingze Gong +5
Multi-person motion prediction is a challenging problem due to the dependency of motion on both individual past movements and interactions with other people. Transformer-based meth…
Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Data
Chaoyi Wu, Xiaoman Zhang, Ya Zhang +2
In this study, we aim to initiate the development of Radiology Foundation Model, termed as RadFM. We consider the construction of foundational models from three perspectives, namel…
arXiVeri: Automatic table verification with GPT
Gyungin Shin, Weidi Xie, Samuel Albanie
Without accurate transcription of numerical data in scientific documents, a scientist cannot draw accurate conclusions. Unfortunately, the process of copying numerical data from on…
Multi-Modal Classifiers for Open-Vocabulary Object Detection
Prannay Kaul, Weidi Xie, Andrew Zisserman
The goal of this paper is open-vocabulary object detection (OVOD) $\unicode{x2013}$ building a model that can detect objects beyond the set of categories seen at training, thus ena…
Boost Video Frame Interpolation via Motion Adaptation
Haoning Wu, Xiaoyun Zhang, Weidi Xie +2
Video frame interpolation (VFI) is a challenging task that aims to generate intermediate frames between two consecutive frames in a video. Existing learning-based VFI methods have…
Zero-shot Composed Text-Image Retrieval
Yikun Liu, Jiangchao Yao, Ya Zhang +2
In this paper, we consider the problem of composed image retrieval (CIR), it aims to train a model that can fuse multi-modal information, e.g., text and images, to accurately retri…
Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Models
Chang Liu, Haoning Wu, Yujie Zhong +3
Generative models have recently exhibited exceptional capabilities in text-to-image generation, but still struggle to generate image sequences coherently. In this work, we focus on…
Annotation-free Audio-Visual Segmentation
Jinxiang Liu, Yu Wang, Chen Ju +3
The objective of Audio-Visual Segmentation (AVS) is to localise the sounding objects within visual scenes by accurately predicting pixel-wise segmentation masks. To tackle the task…
PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering
Xiaoman Zhang, Chaoyi Wu, Ziheng Zhao +4
Medical Visual Question Answering (MedVQA) presents a significant opportunity to enhance diagnostic accuracy and healthcare delivery by leveraging artificial intelligence to interp…
Zero-shot Unsupervised Transfer Instance Segmentation
Gyungin Shin, Samuel Albanie, Weidi Xie
Segmentation is a core computer vision competency, with applications spanning a broad range of scientifically and economically valuable domains. To date, however, the prohibitive c…
PMC-LLaMA: Towards Building Open-source Language Models for Medicine
Chaoyi Wu, Weixiong Lin, Xiaoman Zhang +3
Recently, Large Language Models (LLMs) have showcased remarkable capabilities in natural language understanding. While demonstrating proficiency in everyday conversations and quest…
Towards Open-Vocabulary Video Instance Segmentation
Haochen Wang, Cilin Yan, Shuai Wang +5
Video Instance Segmentation (VIS) aims at segmenting and categorizing objects in videos from a closed set of training categories, lacking the generalization ability to handle novel…
AutoAD: Movie Description in Context
Tengda Han, Max Bain, Arsha Nagrani +3
The objective of this paper is an automatic Audio Description (AD) model that ingests movies and outputs AD in text form. Generating high-quality movie AD is challenging due to the…
Collaboration Helps Camera Overtake LiDAR in 3D Detection
Yue Hu, Yifan Lu, Runsheng Xu +3
Camera-only 3D detection provides an economical solution with a simple configuration for localizing objects in 3D space compared to LiDAR-based detection systems. However, a major…
Multi-modal Prompting for Low-Shot Temporal Action Localization
Chen Ju, Zeqian Li, Peisen Zhao +5
In this paper, we consider the problem of temporal action localization under low-shot (zero-shot & few-shot) scenario, with the goal of detecting and classifying the action instanc…
PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documents
Weixiong Lin, Ziheng Zhao, Xiaoman Zhang +4
Foundation models trained on large-scale dataset gain a recent surge in CV and NLP. In contrast, development in biomedical domain lags far behind due to data scarcity. To address t…
Learning Open-vocabulary Semantic Segmentation Models From Natural Language Supervision
Jilan Xu, Junlin Hou, Yuejie Zhang +4
In this paper, we consider the problem of open-vocabulary semantic segmentation (OVS), which aims to segment objects of arbitrary classes instead of pre-defined, closed-set categor…