64 citations · 133 across the 18 of their papers we have counts for
18 papers
MER-Bench: A Comprehensive Benchmark for Multimodal Meme Reappraisal
Yiqi Nie, Fei Wang, Junjie Chen +5
Memes represent a tightly coupled, multimodal form of social expression, in which visual context and overlaid text jointly convey nuanced affect and commentary. Inspired by cogniti…
EmoSEM: Segment and Explain Emotion Stimuli in Visual Art
Jing Zhang, Dan Guo, Zhangbin Li +1
This paper focuses on a key challenge in visual emotion understanding: given an art image, the model pinpoints pixel regions that trigger a specific human emotion, and generates li…
Label-anticipated Event Disentanglement for Audio-Visual Video Parsing
Jinxing Zhou, Dan Guo, Yuxin Mao +3
Audio-Visual Video Parsing (AVVP) task aims to detect and temporally locate events within audio and visual modalities. Multiple events can overlap in the timeline, making identific…
Advancing Weakly-Supervised Audio-Visual Video Parsing via Segment-wise Pseudo Labeling
Jinxing Zhou, Dan Guo, Yiran Zhong +1
The Audio-Visual Video Parsing task aims to identify and temporally localize the events that occur in either or both the audio and visual streams of audible videos. It often perfor…
Training A Small Emotional Vision Language Model for Visual Art Comprehension
Jing Zhang, Liang Zheng, Meng Wang +1
This paper develops small vision language models to understand visual art, which, given an art work, aims to identify its emotion category and explain this prediction with natural…
Object-aware Adaptive-Positivity Learning for Audio-Visual Question Answering
Zhangbin Li, Dan Guo, Jinxing Zhou +2
This paper focuses on the Audio-Visual Question Answering (AVQA) task that aims to answer questions derived from untrimmed audible videos. To generate accurate answers, an AVQA mod…
Dual-Path Temporal Map Optimization for Make-up Temporal Video Grounding
Jiaxiu Li, Kun Li, Jia Li +3
Make-up temporal video grounding (MTVG) aims to localize the target video segment which is semantically related to a sentence describing a make-up activity, given a long video. Com…
Exploiting Diverse Feature for Multimodal Sentiment Analysis
Jia Li, Wei Qian, Kun Li +3
In this paper, we present our solution to the MuSe-Personalisation sub-challenge in the MuSe 2023 Multimodal Sentiment Analysis Challenge. The task of MuSe-Personalisation aims to…
Dual-path TokenLearner for Remote Photoplethysmography-based Physiological Measurement with Facial Videos
Wei Qian, Dan Guo, Kun Li +2
Remote photoplethysmography (rPPG) based physiological measurement is an emerging yet crucial vision task, whose challenge lies in exploring accurate rPPG prediction from facial vi…
ViGT: Proposal-free Video Grounding with Learnable Token in Transformer
Kun Li, Dan Guo, Meng Wang
The video grounding (VG) task aims to locate the queried action or event in an untrimmed video based on rich linguistic descriptions. Existing proposal-free methods are trapped in…
Data Augmentation for Human Behavior Analysis in Multi-Person Conversations
Kun Li, Dan Guo, Guoliang Chen +2
In this paper, we present the solution of our team HFUT-VUT for the MultiMediate Grand Challenge 2023 at ACM Multimedia 2023. The solution covers three sub-challenges: bodily behav…
Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classification
Kun Li, Dan Guo, Guoliang Chen +2
In this paper, we briefly introduce the solution of our team HFUT-VUT for the Micros-gesture Classification in the MiGA challenge at IJCAI 2023. The micro-gesture classification ta…
Improving Audio-Visual Video Parsing with Pseudo Visual Labels
Jinxing Zhou, Dan Guo, Yiran Zhong +1
Audio-Visual Video Parsing is a task to predict the events that occur in video segments for each modality. It often performs in a weakly supervised manner, where only video event l…
Audio-Visual Segmentation with Semantics
Jinxing Zhou, Xuyang Shen, Jianyuan Wang +8
We propose a new problem called audio-visual segmentation (AVS), in which the goal is to output a pixel-level map of the object(s) that produce sound at the time of the image frame…
Contrastive Positive Sample Propagation along the Audio-Visual Event Line
Jinxing Zhou, Dan Guo, Meng Wang
Visual and audio signals often coexist in natural environments, forming audio-visual events (AVEs). Given a video, we aim to localize video segments containing an AVE and identify…
Emotion Separation and Recognition from a Facial Expression by Generating the Poker Face with Vision Transformers
Jia Li, Jiantao Nie, Dan Guo +2
Representation learning and feature disentanglement have garnered significant research interest in the field of facial expression recognition (FER). The inherent ambiguity of emoti…
Audio-Visual Segmentation
Jinxing Zhou, Jianyuan Wang, Jiayi Zhang +7
We propose to explore a new problem called audio-visual segmentation (AVS), in which the goal is to output a pixel-level map of the object(s) that produce sound at the time of the…
Recurrent Relational Memory Network for Unsupervised Image Captioning
Dan Guo, Yang Wang, Peipei Song +1
Unsupervised image captioning with no annotations is an emerging challenge in computer vision, where the existing arts usually adopt GAN (Generative Adversarial Networks) models. I…