253 citations · 580 across the 27 of their papers we have counts for
28 papers
Can You Predict the Next Mind? Toward Multi-Turn Multi-Party Mental World Modeling
Jiahao Huang, Zheng Lian
While multimodal foundation models have achieved remarkable progress in physical world modeling, generation, and embodied reasoning, their capacity for dynamic mental world modelin…
HiCMAE: Hierarchical Contrastive Masked Autoencoder for Self-Supervised Audio-Visual Emotion Recognition
Licai Sun, Zheng Lian, Bin Liu +1
Audio-Visual Emotion Recognition (AVER) has garnered increasing attention in recent years for its critical role in creating emotion-ware intelligent machines. Previous efforts in t…
MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition
Zheng Lian, Licai Sun, Yong Ren +5
Multimodal emotion recognition plays a crucial role in enhancing user experience in human-computer interaction. Over the past few decades, researchers have proposed a series of alg…
ITEACH-Net: Inverted Teacher-studEnt seArCH Network for Emotion Recognition in Conversation
Haiyang Sun, Zheng Lian, Chenglong Wang +4
There remain two critical challenges that hinder the development of ERC. Firstly, there is a lack of exploration into mining deeper insights from the data itself for conversational…
GPT-4V with Emotion: A Zero-shot Benchmark for Generalized Emotion Recognition
Zheng Lian, Licai Sun, Haiyang Sun +5
Recently, GPT-4 with Vision (GPT-4V) has demonstrated remarkable visual capabilities across various tasks, but its performance in emotion recognition has not been fully evaluated.…
MAE-DFER: Efficient Masked Autoencoder for Self-supervised Dynamic Facial Expression Recognition
Licai Sun, Zheng Lian, Bin Liu +1
Dynamic facial expression recognition (DFER) is essential to the development of intelligent and empathetic machines. Prior efforts in this field mainly fall into supervised learnin…
Explainable Multimodal Emotion Recognition
Zheng Lian, Haiyang Sun, Licai Sun +13
Multimodal emotion recognition is an important research topic in artificial intelligence, whose main goal is to integrate multimodal clues to identify human emotional states. Curre…
MFSN: Multi-perspective Fusion Search Network For Pre-training Knowledge in Speech Emotion Recognition
Haiyang Sun, Fulin Zhang, Yingying Gao +3
Speech Emotion Recognition (SER) is an important research topic in human-computer interaction. Many recent works focus on directly extracting emotional cues through pre-trained kno…
ADD 2023: the Second Audio Deepfake Detection Challenge
Jiangyan Yi, Jianhua Tao, Ruibo Fu +15
Audio deepfake detection is an emerging topic in the artificial intelligence community. The second Audio Deepfake Detection Challenge (ADD 2023) aims to spur researchers around the…
MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learning
Zheng Lian, Haiyang Sun, Licai Sun +15
The first Multimodal Emotion Recognition Challenge (MER 2023) was successfully held at ACM Multimedia. The challenge focuses on system robustness and consists of three distinct tra…
Pseudo Labels Regularization for Imbalanced Partial-Label Learning
Mingyu Xu, Zheng Lian
Partial-label learning (PLL) is an important branch of weakly supervised learning where the single ground truth resides in a set of candidate labels, while the research rarely cons…
VRA: Variational Rectified Activation for Out-of-distribution Detection
Mingyu Xu, Zheng Lian, Bin Liu +1
Out-of-distribution (OOD) detection is critical to building reliable machine learning systems in the open world. Researchers have proposed various strategies to reduce model overco…
ALIM: Adjusting Label Importance Mechanism for Noisy Partial Label Learning
Mingyu Xu, Zheng Lian, Lei Feng +2
Noisy partial label learning (noisy PLL) is an important branch of weakly supervised learning. Unlike PLL where the ground-truth label must conceal in the candidate label set, nois…
IRNet: Iterative Refinement Network for Noisy Partial Label Learning
Zheng Lian, Mingyu Xu, Lan Chen +4
Partial label learning (PLL) is a typical weakly supervised learning, where each sample is associated with a set of candidate labels. Its basic assumption is that the ground-truth…
Efficient Multimodal Transformer with Dual-Level Feature Restoration for Robust Multimodal Sentiment Analysis
Licai Sun, Zheng Lian, Bin Liu +1
With the proliferation of user-generated online videos, Multimodal Sentiment Analysis (MSA) has attracted increasing attention recently. Despite significant progress, there are sti…
Two-Aspect Information Fusion Model For ABAW4 Multi-task Challenge
Haiyang Sun, Zheng Lian, Bin Liu +3
In this paper, we propose the solution to the Multi-Task Learning (MTL) Challenge of the 4th Affective Behavior Analysis in-the-wild (ABAW) competition. The task of ABAW is to pred…
EmotionNAS: Two-stream Neural Architecture Search for Speech Emotion Recognition
Haiyang Sun, Zheng Lian, Bin Liu +6
Speech emotion recognition (SER) is an important research topic in human-computer interaction. Existing works mainly rely on human expertise to design models. Despite their success…
GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversation
Zheng Lian, Lan Chen, Licai Sun +2
Conversations have become a critical data format on social media platforms. Understanding conversation from emotion, content and other aspects also attracts increasing attention fr…
Cross Modification Attention Based Deliberation Model for Image Captioning
Zheng Lian, Yanan Zhang, Haichang Li +2
The conventional encoder-decoder framework for image captioning generally adopts a single-pass decoding process, which predicts the target descriptive sentence word by word in temp…
Domain adversarial learning for emotion recognition
Zheng Lian, Jianhua Tao, Bin Liu +1
In practical applications for emotion recognition, users do not always exist in the training corpus. The mismatch between training speakers and testing speakers affects the perform…
Unsupervised Representation Learning with Future Observation Prediction for Speech Emotion Recognition
Zheng Lian, Jianhua Tao, Bin Liu +1
Prior works on speech emotion recognition utilize various unsupervised learning approaches to deal with low-resource samples. However, these methods pay less attention to modeling…
Conversational Emotion Analysis via Attention Mechanisms
Zheng Lian, Jianhua Tao, Bin Liu +1
Different from the emotion recognition in individual utterances, we propose a multimodal learning framework using relation and dependencies among the utterances for conversational…
Expression Analysis Based on Face Regions in Read-world Conditions
Zheng Lian, Ya Li, Jian-Hua Tao +2
Facial emotion recognition is an essential and important aspect of the field of human-machine interaction. Past research on facial emotion recognition focuses on the laboratory env…
Speech Emotion Recognition via Contrastive Loss under Siamese Networks
Zheng Lian, Ya Li, Jianhua Tao +1
Speech emotion recognition is an important aspect of human-computer interaction. Prior work proposes various end-to-end models to improve the classification performance. However, m…