373 citations · 1k across the 52 of their papers we have counts for
43 papers · 1 filter
Multimodal Variational Auto-encoder based Audio-Visual Segmentation
Yuxin Mao, Jing Zhang, Mochu Xiang +2
We propose an Explicit Conditional Multimodal Variational Auto-Encoder (ECMVAE) for audio-visual segmentation (AVS), aiming to segment sound sources in the video sequence. Existing…
All-pairs Consistency Learning for Weakly Supervised Semantic Segmentation
Weixuan Sun, Yanhao Zhang, Zhen Qin +5
In this work, we propose a new transformer-based regularization to better localize objects for Weakly supervised semantic segmentation (WSSS). In image-level WSSS, Class Activation…
Improving Audio-Visual Segmentation with Bidirectional Generation
Dawei Hao, Yuxin Mao, Bowen He +3
The aim of audio-visual segmentation (AVS) is to precisely differentiate audible objects within videos down to the pixel level. Traditional approaches often tackle this challenge b…
Image-based Geolocalization by Ground-to-2.5D Map Matching
Mengjie Zhou, Liu Liu, Yiran Zhong +1
We study the image-based geolocalization problem, aiming to localize ground-view query images on cartographic maps. Current methods often utilize cross-view localization techniques…
Joint Salient Object Detection and Camouflaged Object Detection via Uncertainty-aware Learning
Aixuan Li, Jing Zhang, Yunqiu Lv +4
Salient objects attract human attention and usually stand out clearly from their surroundings. In contrast, camouflaged objects share similar colors or textures with the environmen…
Contrastive Conditional Latent Diffusion for Audio-visual Segmentation
Yuxin Mao, Jing Zhang, Mochu Xiang +4
We propose a contrastive conditional latent diffusion model for audio-visual segmentation (AVS) to thoroughly investigate the impact of audio, where the correlation between audio a…
An Alternative to WSSS? An Empirical Study of the Segment Anything Model (SAM) on Weakly-Supervised Semantic Segmentation Problems
Weixuan Sun, Zheyuan Liu, Yanhao Zhang +2
The Segment Anything Model (SAM) has demonstrated exceptional performance and versatility, making it a promising tool for various related tasks. In this report, we explore the appl…
Fine-grained Audible Video Description
Xuyang Shen, Dong Li, Jinxing Zhou +9
We explore a new task for audio-visual-language modeling called fine-grained audible video description (FAVD). It aims to provide detailed textual descriptions for the given audibl…
Learning Audio-Visual Source Localization via False Negative Aware Contrastive Learning
Weixuan Sun, Jiayi Zhang, Jianyuan Wang +6
Self-supervised audio-visual source localization aims to locate sound-source objects in video frames without extra annotations. Recent methods often approach this goal with the hel…
Improving Audio-Visual Video Parsing with Pseudo Visual Labels
Jinxing Zhou, Dan Guo, Yiran Zhong +1
Audio-Visual Video Parsing is a task to predict the events that occur in video segments for each modality. It often performs in a weakly supervised manner, where only video event l…
Audio-Visual Segmentation with Semantics
Jinxing Zhou, Xuyang Shen, Jianyuan Wang +8
We propose a new problem called audio-visual segmentation (AVS), in which the goal is to output a pixel-level map of the object(s) that produce sound at the time of the image frame…
Linear Video Transformer with Feature Fixation
Kaiyue Lu, Zexiang Liu, Jianyuan Wang +8
Vision Transformers have achieved impressive performance in video classification, while suffering from the quadratic complexity caused by the Softmax attention mechanism. Some stud…
AMF: Adaptable Weighting Fusion with Multiple Fine-tuning for Image Classification
Xuyang Shen, Jo Plested, Sabrina Caldwell +2
Fine-tuning is widely applied in image classification tasks as a transfer learning approach. It re-uses the knowledge from a source task to learn and obtain a high performance in t…
Deep Laparoscopic Stereo Matching with Transformers
Xuelian Cheng, Yiran Zhong, Mehrtash Harandi +3
The self-attention mechanism, successfully employed with the transformer structure is shown promise in many computer vision tasks including image recognition, and object detection.…
Audio-Visual Segmentation
Jinxing Zhou, Jianyuan Wang, Jiayi Zhang +7
We propose to explore a new problem called audio-visual segmentation (AVS), in which the goal is to output a pixel-level map of the object(s) that produce sound at the time of the…
Vicinity Vision Transformer
Weixuan Sun, Zhen Qin, Hui Deng +7
Vision transformers have shown great success on numerous computer vision tasks. However, its central component, softmax attention, prohibits vision transformers from scaling up to…
Deep Non-rigid Structure-from-Motion: A Sequence-to-Sequence Translation Perspective
Hui Deng, Tong Zhang, Yuchao Dai +3
Directly regressing the non-rigid shape and camera pose from the individual 2D frame is ill-suited to the Non-Rigid Structure-from-Motion (NRSfM) problem. This frame-by-frame 3D re…
Implicit Motion Handling for Video Camouflaged Object Detection
Xuelian Cheng, Huan Xiong, Deng-Ping Fan +4
We propose a new video camouflaged object detection (VCOD) framework that can exploit both short-term dynamics and long-term temporal consistency to detect camouflaged objects from…
GETAM: Gradient-weighted Element-wise Transformer Attention Map for Weakly-supervised Semantic segmentation
Weixuan Sun, Jing Zhang, Zheyuan Liu +2
Weakly Supervised Semantic Segmentation (WSSS) is challenging, particularly when image-level labels are used to supervise pixel level prediction. To bridge their gap, a Class Activ…
MUNet: Motion Uncertainty-aware Semi-supervised Video Object Segmentation
Jiadai Sun, Yuxin Mao, Yuchao Dai +2
The task of semi-supervised video object segmentation (VOS) has been greatly advanced and state-of-the-art performance has been made by dense matching-based methods. The recent met…
Dense Uncertainty Estimation via an Ensemble-based Conditional Latent Variable Model
Jing Zhang, Yuchao Dai, Mehrtash Harandi +3
Uncertainty estimation has been extensively studied in recent literature, which can usually be classified as aleatoric uncertainty and epistemic uncertainty. In current aleatoric u…
RGB-D Saliency Detection via Cascaded Mutual Information Minimization
Jing Zhang, Deng-Ping Fan, Yuchao Dai +4
Existing RGB-D saliency detection models do not explicitly encourage RGB and depth to achieve effective multi-modal learning. In this paper, we introduce a novel multi-stage cascad…
Memory-Free Generative Replay For Class-Incremental Learning
Xiaomeng Xin, Yiran Zhong, Yunzhong Hou +2
Regularization-based methods are beneficial to alleviate the catastrophic forgetting problem in class-incremental learning. With the absence of old task images, they often assume t…
Invertible Attention
Jiajun Zha, Yiran Zhong, Jing Zhang +2
Attention has been proved to be an efficient mechanism to capture long-range dependencies. However, so far it has not been deployed in invertible networks. This is due to the fact…