253 citations · 394 across the 22 of their papers we have counts for
22 papers
Human-JEPA: A Human-Centric Vision Model that Perceives and Anticipates
Hui Wei, Licai Sun, Guoying Zhao
Machines that understand humans should perceive the present and anticipate the future. Existing human-centric vision model are pretrained on human images, set the state of the art…
Learning Transferable Facial Emotion Representations from Large-Scale Semantically Rich Captions
Licai Sun, Xingxun Jiang, Haoyu Chen +7
Current facial emotion recognition systems are predominately trained to predict a fixed set of predefined categories or abstract dimensional values. This constrained form of superv…
EmoPrefer: Can Large Language Models Understand Human Emotion Preferences?
Zheng Lian, Licai Sun, Lan Chen +8
Descriptive Multimodal Emotion Recognition (DMER) has garnered increasing research attention. Unlike traditional discriminative paradigms that rely on predefined emotion taxonomies…
MagicPortrait: Temporally Consistent Face Reenactment with 3D Geometric Guidance
Mengting Wei, Yante Li, Tuomas Varanka +2
In this study, we propose a method for video face reenactment that integrates a 3D face parametric model into a latent diffusion framework, aiming to improve shape consistency and…
AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models
Zheng Lian, Haoyu Chen, Lan Chen +9
The emergence of multimodal large language models (MLLMs) advances multimodal emotion recognition (MER) to the next level, from naive discriminative tasks to complex emotion unders…
OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognition
Zheng Lian, Haiyang Sun, Licai Sun +13
Multimodal Emotion Recognition (MER) is a critical research area that seeks to decode human emotions from diverse data modalities. However, existing machine learning methods predom…
AffectGPT: Dataset and Framework for Explainable Multimodal Emotion Recognition
Zheng Lian, Haiyang Sun, Licai Sun +3
Explainable Multimodal Emotion Recognition (EMER) is an emerging task that aims to achieve reliable and accurate emotion recognition. However, due to the high annotation cost, the…
MER 2024: Semi-Supervised Learning, Noise Robustness, and Open-Vocabulary Multimodal Emotion Recognition
Zheng Lian, Haiyang Sun, Licai Sun +15
Multimodal emotion recognition is an important research topic in artificial intelligence. Over the past few decades, researchers have made remarkable progress by increasing the dat…
Multimodal Fusion with Pre-Trained Model Features in Affective Behaviour Analysis In-the-wild
Zhuofan Wen, Fengyu Zhang, Siyuan Zhang +6
Multimodal fusion is a significant method for most multimodal tasks. With the recent surge in the number of large pre-trained models, combining both multimodal fusion methods and p…
HiCMAE: Hierarchical Contrastive Masked Autoencoder for Self-Supervised Audio-Visual Emotion Recognition
Licai Sun, Zheng Lian, Bin Liu +1
Audio-Visual Emotion Recognition (AVER) has garnered increasing attention in recent years for its critical role in creating emotion-ware intelligent machines. Previous efforts in t…
MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition
Zheng Lian, Licai Sun, Yong Ren +5
Multimodal emotion recognition plays a crucial role in enhancing user experience in human-computer interaction. Over the past few decades, researchers have proposed a series of alg…
SVFAP: Self-supervised Video Facial Affect Perceiver
Licai Sun, Zheng Lian, Kexin Wang +5
Video-based facial affect analysis has recently attracted increasing attention owing to its critical role in human-computer interaction. Previous studies mainly focus on developing…
ITEACH-Net: Inverted Teacher-studEnt seArCH Network for Emotion Recognition in Conversation
Haiyang Sun, Zheng Lian, Chenglong Wang +4
There remain two critical challenges that hinder the development of ERC. Firstly, there is a lack of exploration into mining deeper insights from the data itself for conversational…
GPT-4V with Emotion: A Zero-shot Benchmark for Generalized Emotion Recognition
Zheng Lian, Licai Sun, Haiyang Sun +5
Recently, GPT-4 with Vision (GPT-4V) has demonstrated remarkable visual capabilities across various tasks, but its performance in emotion recognition has not been fully evaluated.…
MAE-DFER: Efficient Masked Autoencoder for Self-supervised Dynamic Facial Expression Recognition
Licai Sun, Zheng Lian, Bin Liu +1
Dynamic facial expression recognition (DFER) is essential to the development of intelligent and empathetic machines. Prior efforts in this field mainly fall into supervised learnin…
Explainable Multimodal Emotion Recognition
Zheng Lian, Haiyang Sun, Licai Sun +13
Multimodal emotion recognition is an important research topic in artificial intelligence, whose main goal is to integrate multimodal clues to identify human emotional states. Curre…
MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learning
Zheng Lian, Haiyang Sun, Licai Sun +15
The first Multimodal Emotion Recognition Challenge (MER 2023) was successfully held at ACM Multimedia. The challenge focuses on system robustness and consists of three distinct tra…
IRNet: Iterative Refinement Network for Noisy Partial Label Learning
Zheng Lian, Mingyu Xu, Lan Chen +4
Partial label learning (PLL) is a typical weakly supervised learning, where each sample is associated with a set of candidate labels. Its basic assumption is that the ground-truth…
Efficient Multimodal Transformer with Dual-Level Feature Restoration for Robust Multimodal Sentiment Analysis
Licai Sun, Zheng Lian, Bin Liu +1
With the proliferation of user-generated online videos, Multimodal Sentiment Analysis (MSA) has attracted increasing attention recently. Despite significant progress, there are sti…
Two-Aspect Information Fusion Model For ABAW4 Multi-task Challenge
Haiyang Sun, Zheng Lian, Bin Liu +3
In this paper, we propose the solution to the Multi-Task Learning (MTL) Challenge of the 4th Affective Behavior Analysis in-the-wild (ABAW) competition. The task of ABAW is to pred…
EmotionNAS: Two-stream Neural Architecture Search for Speech Emotion Recognition
Haiyang Sun, Zheng Lian, Bin Liu +6
Speech emotion recognition (SER) is an important research topic in human-computer interaction. Existing works mainly rely on human expertise to design models. Despite their success…
GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversation
Zheng Lian, Lan Chen, Licai Sun +2
Conversations have become a critical data format on social media platforms. Understanding conversation from emotion, content and other aspects also attracts increasing attention fr…