167 citations · 272 across the 17 of their papers we have counts for
17 papers
Addressing Missing and Noisy Modalities in One Solution: Unified Modality-Quality Framework for Low-quality Multimodal Data
Sijie Mai, Shiqin Han, Haifeng Hu
Multimodal data encountered in real-world scenarios are typically of low quality, with noisy modalities and missing modalities being typical forms that severely hinder model perfor…
CyIN: Cyclic Informative Latent Space for Bridging Complete and Incomplete Multimodal Learning
Ronghao Lin, Qiaolin He, Sijie Mai +5
Multimodal machine learning, mimicking the human brain's ability to integrate various modalities has seen rapid growth. Most previous multimodal models are trained on perfectly pai…
MissMAC-Bench: Building Solid Benchmark for Missing Modality Issue in Robust Multimodal Affective Computing
Ronghao Lin, Honghao Lu, Ruixing Wu +5
Current Multimodal Affective Computing (MAC) systems heavily rely on the completeness of multiple modalities to accurately understand human's affective state. However, in real-worl…
Multi-source Multimodal Progressive Domain Adaption for Audio-Visual Deception Detection
Ronghao Lin, Sijie Mai, Ying Zeng +3
This paper presents the winning approach for the 1st MultiModal Deception Detection (MMDD) Challenge at the 1st Workshop on Subtle Visual Computing (SVC). Aiming at the domain shif…
Meta-Learn Unimodal Signals with Weak Supervision for Multimodal Sentiment Analysis
Sijie Mai, Yu Zhao, Ying Zeng +2
Multimodal sentiment analysis aims to effectively integrate information from various sources to infer sentiment, where in many cases there are no annotations for unimodal labels. T…
End-to-end Semantic-centric Video-based Multimodal Affective Computing
Ronghao Lin, Ying Zeng, Sijie Mai +1
In the pathway toward Artificial General Intelligence (AGI), understanding human's affection is essential to enhance machine's cognition abilities. For achieving more sensual human…
Curriculum Learning Meets Weakly Supervised Modality Correlation Learning
Sijie Mai, Ya Sun, Haifeng Hu
In the field of multimodal sentiment analysis (MSA), a few studies have leveraged the inherent modality correlation information stored in samples for self-supervised learning. Howe…
Relation-dependent Contrastive Learning with Cluster Sampling for Inductive Relation Prediction
Jianfeng Wu, Sijie Mai, Haifeng Hu
Relation prediction is a task designed for knowledge graph completion which aims to predict missing relationships between entities. Recent subgraph-based models for inductive relat…
Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representations
Sijie Mai, Ying Zeng, Haifeng Hu
Learning effective joint embedding for cross-modal data has always been a focus in the field of multimodal machine learning. We argue that during multimodal fusion, the generated m…
Communicative Subgraph Representation Learning for Multi-Relational Inductive Drug-Gene Interaction Prediction
Jiahua Rao, Shuangjia Zheng, Sijie Mai +1
Illuminating the interconnections between drugs and genes is an important topic in drug development and precision medicine. Currently, computational predictions of drug-gene intera…
Which is Making the Contribution: Modulating Unimodal and Cross-modal Dynamics for Multimodal Sentiment Analysis
Ying Zeng, Sijie Mai, Haifeng Hu
Multimodal sentiment analysis (MSA) draws increasing attention with the availability of multimodal data. The boost in performance of MSA models is mainly hindered by two problems.…
Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysis
Sijie Mai, Ying Zeng, Shuangjia Zheng +1
The wide application of smart devices enables the availability of multimodal data, which can be utilized in many tasks. In the field of multimodal sentiment analysis (MSA), most pr…
Graph Capsule Aggregation for Unaligned Multimodal Sequences
Jianfeng Wu, Sijie Mai, Haifeng Hu
Humans express their opinions and emotions through multiple modalities which mainly consist of textual, acoustic and visual modalities. Prior works on multimodal sentiment analysis…
Subgraph-aware Few-Shot Inductive Link Prediction via Meta-Learning
Shuangjia Zheng, Sijie Mai, Ya Sun +2
Link prediction for knowledge graphs aims to predict missing connections between entities. Prevailing methods are limited to a transductive setting and hard to process unseen entit…
Communicative Message Passing for Inductive Relation Reasoning
Sijie Mai, Shuangjia Zheng, Yuedong Yang +1
Relation prediction for knowledge graphs aims at predicting missing relationships between entities. Despite the importance of inductive relation prediction, most previous works are…
Analyzing Unaligned Multimodal Sequence via Graph Convolution and Graph Pooling Fusion
Sijie Mai, Songlong Xing, Jiaxuan He +2
In this paper, we study the task of multimodal sequence analysis which aims to draw inferences from visual, language and acoustic sequences. A majority of existing works generally…
Modality to Modality Translation: An Adversarial Representation Learning and Graph Fusion Network for Multimodal Fusion
Sijie Mai, Haifeng Hu, Songlong Xing
Learning joint embedding space for various modalities is of vital importance for multimodal fusion. Mainstream modality fusion approaches fail to achieve this goal, leaving a modal…