170 citations · 209 across the 22 of their papers we have counts for
22 papers
MER-Bench: A Comprehensive Benchmark for Multimodal Meme Reappraisal
Yiqi Nie, Fei Wang, Junjie Chen +5
Memes represent a tightly coupled, multimodal form of social expression, in which visual context and overlaid text jointly convey nuanced affect and commentary. Inspired by cogniti…
Face-Guided Sentiment Boundary Enhancement for Weakly-Supervised Temporal Sentiment Localization
Cailing Han, Zhangbin Li, Jinxing Zhou +5
Point-level weakly-supervised temporal sentiment localization (P-WTSL) aims to detect sentiment-relevant segments in untrimmed multimodal videos using timestamp sentiment annotatio…
SSAM: Self-Supervised Association Modeling for Test-Time Adaption
Yaxiong Wang, Zhenqiang Zhang, Lechao Cheng +3
Test-time adaption (TTA) has witnessed important progress in recent years, the prevailing methods typically first encode the image and the text and design strategies to model the a…
Towards Efficient Partially Relevant Video Retrieval with Active Moment Discovering
Peipei Song, Long Zhang, Long Lan +4
Partially relevant video retrieval (PRVR) is a practical yet challenging task in text-to-video retrieval, where videos are untrimmed and contain much background content. The pursui…
EmoSEM: Segment and Explain Emotion Stimuli in Visual Art
Jing Zhang, Dan Guo, Zhangbin Li +1
This paper focuses on a key challenge in visual emotion understanding: given an art image, the model pinpoints pixel regions that trigger a specific human emotion, and generates li…
EgoTextVQA: Towards Egocentric Scene-Text Aware Video Question Answering
Sheng Zhou, Junbin Xiao, Qingyun Li +6
We introduce EgoTextVQA, a novel and rigorously constructed benchmark for egocentric QA assistance involving scene text. EgoTextVQA contains 1.5K ego-view videos and 7K scene-text…
Repetitive Action Counting with Hybrid Temporal Relation Modeling
Kun Li, Xinge Peng, Dan Guo +2
Repetitive Action Counting (RAC) aims to count the number of repetitive actions occurring in videos. In the real world, repetitive actions have great diversity and bring numerous c…
MOL-Mamba: Enhancing Molecular Representation with Structural & Electronic Insights
Jingjing Hu, Dan Guo, Zhan Si +5
Molecular representation learning plays a crucial role in various downstream tasks, such as molecular property prediction and drug design. To accurately represent molecules, Graph…
ASAP: Advancing Semantic Alignment Promotes Multi-Modal Manipulation Detecting and Grounding
Zhenxing Zhang, Yaxiong Wang, Lechao Cheng +3
We present ASAP, a new framework for detecting and grounding multi-modal media manipulation (DGM4).Upon thorough examination, we observe that accurate fine-grained cross-modal sema…
Grounding is All You Need? Dual Temporal Grounding for Video Dialog
You Qin, Wei Ji, Xinze Lan +5
In the realm of video dialog response generation, the understanding of video content and the temporal nuances of conversation history are paramount. While a segment of current rese…
Towards Open-Vocabulary Audio-Visual Event Localization
Jinxing Zhou, Dan Guo, Ruohao Guo +5
The Audio-Visual Event Localization (AVEL) task aims to temporally locate and classify video events that are both audible and visible. Most research in this field assumes a closed-…
Scene-Text Grounding for Text-Based Video Question Answering
Sheng Zhou, Junbin Xiao, Xun Yang +5
Existing efforts in text-based video question answering (TextVideoQA) are criticized for their opaque decisionmaking and heavy reliance on scene-text recognition. In this paper, we…
Label-anticipated Event Disentanglement for Audio-Visual Video Parsing
Jinxing Zhou, Dan Guo, Yuxin Mao +3
Audio-Visual Video Parsing (AVVP) task aims to detect and temporally locate events within audio and visual modalities. Multiple events can overlap in the timeline, making identific…
Joint Spatial-Temporal Modeling and Contrastive Learning for Self-supervised Heart Rate Measurement
Wei Qian, Qi Li, Kun Li +4
This paper briefly introduces the solutions developed by our team, HFUT-VUT, for Track 1 of self-supervised heart rate measurement in the 3rd Vision-based Remote Physiological Sign…
Advancing Weakly-Supervised Audio-Visual Video Parsing via Segment-wise Pseudo Labeling
Jinxing Zhou, Dan Guo, Yiran Zhong +1
The Audio-Visual Video Parsing task aims to identify and temporally localize the events that occur in either or both the audio and visual streams of audible videos. It often perfor…
Unified Static and Dynamic Network: Efficient Temporal Filtering for Video Grounding
Jingjing Hu, Dan Guo, Kun Li +4
Inspired by the activity-silent and persistent activity mechanisms in human visual perception biology, we design a Unified Static and Dynamic Network (UniSDNet), to learn the seman…
Training A Small Emotional Vision Language Model for Visual Art Comprehension
Jing Zhang, Liang Zheng, Meng Wang +1
This paper develops small vision language models to understand visual art, which, given an art work, aims to identify its emotion category and explain this prediction with natural…
Benchmarking Micro-action Recognition: Dataset, Methods, and Applications
Dan Guo, Kun Li, Bin Hu +2
Micro-action is an imperceptible non-verbal behaviour characterised by low-intensity movement. It offers insights into the feelings and intentions of individuals and is important f…
Object-aware Adaptive-Positivity Learning for Audio-Visual Question Answering
Zhangbin Li, Dan Guo, Jinxing Zhou +2
This paper focuses on the Audio-Visual Question Answering (AVQA) task that aims to answer questions derived from untrimmed audible videos. To generate accurate answers, an AVQA mod…
EulerMormer: Robust Eulerian Motion Magnification via Dynamic Filtering within Transformer
Fei Wang, Dan Guo, Kun Li +1
Video Motion Magnification (VMM) aims to break the resolution limit of human visual perception capability and reveal the imperceptible minor motion that contains valuable informati…
Exploring Sparse Spatial Relation in Graph Inference for Text-Based VQA
Sheng Zhou, Dan Guo, Jia Li +2
Text-based visual question answering (TextVQA) faces the significant challenge of avoiding redundant relational inference. To be specific, a large number of detected objects and op…
Dual-Path Temporal Map Optimization for Make-up Temporal Video Grounding
Jiaxiu Li, Kun Li, Jia Li +3
Make-up temporal video grounding (MTVG) aims to localize the target video segment which is semantically related to a sentence describing a make-up activity, given a long video. Com…