49 citations · 144 across the 32 of their papers we have counts for
16 papers · 1 filter
I-KAN: Interpretable Intra- and Inter-Modal Multi-Pathway Representation Learning for Multimodal Sentiment Analysis
Xiaoting Xie, Feng Qiao, Xianwei Zhang +5
Multimodal sentiment analysis (MSA) aims to infer sentiment by integrating linguistic, acoustic, and visual information. However, many fusion methods integrate modality-specific re…
A Comprehensive Review of Multimodal Facial State Analysis: Tasks, Methods, and Resources
Xuri Ge, Tianshuo Zhang, Ruihan Li +6
Facial state analysis plays a crucial role in understanding human expressions, psychological modeling, and human computer interaction. Traditional unimodal vision-based methods are…
MCoT-MVS: Multi-level Vision Selection by Multi-modal Chain-of-Thought Reasoning for Composed Image Retrieval
Xuri Ge, Chunhao Wang, Xindi Wang +3
Composed Image Retrieval (CIR) aims to retrieve target images based on a reference image and modified texts. However, existing methods often struggle to extract the correct semanti…
Hierarchical Dual-Change Collaborative Learning for UAV Scene Change Captioning
Fuhai Chen, Pengpeng Huang, Junwen Wu +4
This paper proposes a novel task for UAV scene understanding - UAV Scene Change Captioning (UAV-SCC) - which aims to generate natural language descriptions of semantic changes in d…
Focal-RegionFace: Generating Fine-Grained Multi-attribute Descriptions for Arbitrarily Selected Face Focal Regions
Kaiwen Zheng, Junchen Fu, Songpei Xu +4
In this paper, we introduce an underexplored problem in facial analysis: generating and recognizing multi-attribute natural language descriptions, containing facial action units (A…
Multimodal Representation Learning Techniques for Comprehensive Facial State Analysis
Kaiwen Zheng, Xuri Ge, Junchen Fu +2
Multimodal foundation models have significantly improved feature representation by integrating information from multiple modalities, making them highly suitable for a broader set o…
Towards End-to-End Explainable Facial Action Unit Recognition via Vision-Language Joint Learning
Xuri Ge, Junchen Fu, Fuhai Chen +3
Facial action units (AUs), as defined in the Facial Action Coding System (FACS), have received significant research interest owing to their diverse range of applications in facial…
Hire: Hybrid-modal Interaction with Multiple Relational Enhancements for Image-Text Matching
Xuri Ge, Fuhai Chen, Songpei Xu +3
Image-text matching (ITM) is a fundamental problem in computer vision. The key issue lies in jointly learning the visual and textual representation to estimate their similarity acc…
Detail-Enhanced Intra- and Inter-modal Interaction for Audio-Visual Emotion Recognition
Tong Shi, Xuri Ge, Joemon M. Jose +2
Capturing complex temporal relationships between video and audio modalities is vital for Audio-Visual Emotion Recognition (AVER). However, existing methods lack attention to local…
3SHNet: Boosting Image-Sentence Retrieval via Visual Semantic-Spatial Self-Highlighting
Xuri Ge, Songpei Xu, Fuhai Chen +4
In this paper, we propose a novel visual Semantic-Spatial Self-Highlighting Network (termed 3SHNet) for high-precision, high-efficiency and high-generalization image-sentence retri…
Cross-modal Semantic Enhanced Interaction for Image-Sentence Retrieval
Xuri Ge, Fuhai Chen, Songpei Xu +2
Image-sentence retrieval has attracted extensive research attention in multimedia and computer vision due to its promising application. The key issue lies in jointly learning the v…
MGRR-Net: Multi-level Graph Relational Reasoning Network for Facial Action Units Detection
Xuri Ge, Joemon M. Jose, Songpei Xu +2
The Facial Action Coding System (FACS) encodes the action units (AUs) in facial images, which has attracted extensive research attention due to its wide use in facial expression an…
Automatic Facial Paralysis Estimation with Facial Action Units
Xuri Ge, Joemon M. Jose, Pengcheng Wang +3
Facial palsy is unilateral facial nerve weakness or paralysis of rapid onset with unknown causes. Automatically estimating facial palsy severeness can be helpful for the diagnosis…
Factored Attention and Embedding for Unstructured-view Topic-related Ultrasound Report Generation
Fuhai Chen, Rongrong Ji, Chengpeng Dai +4
Echocardiography is widely used to clinical practice for diagnosis and treatment, e.g., on the common congenital heart defects. The traditional manual manipulation is error-prone d…
Differentiated Relevances Embedding for Group-based Referring Expression Comprehension
Fuhai Chen, Xuri Ge, Xiaoshuai Sun +4
The key of referring expression comprehension lies in capturing the cross-modal visual-linguistic relevance. Existing works typically model the cross-modal relevance in each image,…
Structured Multi-modal Feature Embedding and Alignment for Image-Sentence Retrieval
Xuri Ge, Fuhai Chen, Joemon M. Jose +3
The current state-of-the-art image-sentence retrieval methods implicitly align the visual-textual fragments, like regions in images and words in sentences, and adopt attention modu…