373 citations · 1k across the 52 of their papers we have counts for
52 papers
Multimodal Variational Auto-encoder based Audio-Visual Segmentation
Yuxin Mao, Jing Zhang, Mochu Xiang +2
We propose an Explicit Conditional Multimodal Variational Auto-Encoder (ECMVAE) for audio-visual segmentation (AVS), aiming to segment sound sources in the video sequence. Existing…
All-pairs Consistency Learning for Weakly Supervised Semantic Segmentation
Weixuan Sun, Yanhao Zhang, Zhen Qin +5
In this work, we propose a new transformer-based regularization to better localize objects for Weakly supervised semantic segmentation (WSSS). In image-level WSSS, Class Activation…
Improving Audio-Visual Segmentation with Bidirectional Generation
Dawei Hao, Yuxin Mao, Bowen He +3
The aim of audio-visual segmentation (AVS) is to precisely differentiate audible objects within videos down to the pixel level. Traditional approaches often tackle this challenge b…
Image-based Geolocalization by Ground-to-2.5D Map Matching
Mengjie Zhou, Liu Liu, Yiran Zhong +1
We study the image-based geolocalization problem, aiming to localize ground-view query images on cartographic maps. Current methods often utilize cross-view localization techniques…
Linearized Relative Positional Encoding
Zhen Qin, Weixuan Sun, Kaiyue Lu +6
Relative positional encoding is widely used in vanilla and linear transformers to represent positional information. However, existing encoding methods of a vanilla transformer are…
Joint Salient Object Detection and Camouflaged Object Detection via Uncertainty-aware Learning
Aixuan Li, Jing Zhang, Yunqiu Lv +4
Salient objects attract human attention and usually stand out clearly from their surroundings. In contrast, camouflaged objects share similar colors or textures with the environmen…
Contrastive Conditional Latent Diffusion for Audio-visual Segmentation
Yuxin Mao, Jing Zhang, Mochu Xiang +4
We propose a contrastive conditional latent diffusion model for audio-visual segmentation (AVS) to thoroughly investigate the impact of audio, where the correlation between audio a…
TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer
Zhen Qin, Dong Li, Weigao Sun +8
We present TransNormerLLM, the first linear attention-based Large Language Model (LLM) that outperforms conventional softmax attention-based models in terms of both accuracy and ef…
Exploring Transformer Extrapolation
Zhen Qin, Yiran Zhong, Hui Deng
Length extrapolation has attracted considerable attention recently since it allows transformers to be tested on longer sequences than those used in training. Previous research has…
An Alternative to WSSS? An Empirical Study of the Segment Anything Model (SAM) on Weakly-Supervised Semantic Segmentation Problems
Weixuan Sun, Zheyuan Liu, Yanhao Zhang +2
The Segment Anything Model (SAM) has demonstrated exceptional performance and versatility, making it a promising tool for various related tasks. In this report, we explore the appl…
Toeplitz Neural Network for Sequence Modeling
Zhen Qin, Xiaodong Han, Weixuan Sun +6
Sequence modeling has important applications in natural language processing and computer vision. Recently, the transformer-based models have shown strong performance on various seq…
Fine-grained Audible Video Description
Xuyang Shen, Dong Li, Jinxing Zhou +9
We explore a new task for audio-visual-language modeling called fine-grained audible video description (FAVD). It aims to provide detailed textual descriptions for the given audibl…
Learning Audio-Visual Source Localization via False Negative Aware Contrastive Learning
Weixuan Sun, Jiayi Zhang, Jianyuan Wang +6
Self-supervised audio-visual source localization aims to locate sound-source objects in video frames without extra annotations. Recent methods often approach this goal with the hel…
Improving Audio-Visual Video Parsing with Pseudo Visual Labels
Jinxing Zhou, Dan Guo, Yiran Zhong +1
Audio-Visual Video Parsing is a task to predict the events that occur in video segments for each modality. It often performs in a weakly supervised manner, where only video event l…
Audio-Visual Segmentation with Semantics
Jinxing Zhou, Xuyang Shen, Jianyuan Wang +8
We propose a new problem called audio-visual segmentation (AVS), in which the goal is to output a pixel-level map of the object(s) that produce sound at the time of the image frame…
The Devil in Linear Transformer
Zhen Qin, XiaoDong Han, Weixuan Sun +4
Linear transformers aim to reduce the quadratic space-time complexity of vanilla transformers. However, they usually suffer from degraded performances on various tasks and corpus.…
Linear Video Transformer with Feature Fixation
Kaiyue Lu, Zexiang Liu, Jianyuan Wang +8
Vision Transformers have achieved impressive performance in video classification, while suffering from the quadratic complexity caused by the Softmax attention mechanism. Some stud…
Neural Architecture Search on Efficient Transformers and Beyond
Zexiang Liu, Dong Li, Kaiyue Lu +4
Recently, numerous efficient Transformers have been proposed to reduce the quadratic computational complexity of standard Transformers caused by the Softmax attention. However, mos…
AMF: Adaptable Weighting Fusion with Multiple Fine-tuning for Image Classification
Xuyang Shen, Jo Plested, Sabrina Caldwell +2
Fine-tuning is widely applied in image classification tasks as a transfer learning approach. It re-uses the knowledge from a source task to learn and obtain a high performance in t…
Deep Laparoscopic Stereo Matching with Transformers
Xuelian Cheng, Yiran Zhong, Mehrtash Harandi +3
The self-attention mechanism, successfully employed with the transformer structure is shown promise in many computer vision tasks including image recognition, and object detection.…
Audio-Visual Segmentation
Jinxing Zhou, Jianyuan Wang, Jiayi Zhang +7
We propose to explore a new problem called audio-visual segmentation (AVS), in which the goal is to output a pixel-level map of the object(s) that produce sound at the time of the…
Vicinity Vision Transformer
Weixuan Sun, Zhen Qin, Hui Deng +7
Vision transformers have shown great success on numerous computer vision tasks. However, its central component, softmax attention, prohibits vision transformers from scaling up to…
Deep Non-rigid Structure-from-Motion: A Sequence-to-Sequence Translation Perspective
Hui Deng, Tong Zhang, Yuchao Dai +3
Directly regressing the non-rigid shape and camera pose from the individual 2D frame is ill-suited to the Non-Rigid Structure-from-Motion (NRSfM) problem. This frame-by-frame 3D re…
Locality Matters: A Locality-Biased Linear Attention for Automatic Speech Recognition
Jingyu Sun, Guiping Zhong, Dinghao Zhou +2
Conformer has shown a great success in automatic speech recognition (ASR) on many public benchmarks. One of its crucial drawbacks is the quadratic time-space complexity with respec…