29 citations · 59 across the 17 of their papers we have counts for
17 papers
OV-InstructTTS: Towards Open-Vocabulary Instruct Text-to-Speech
Yong Ren, Jiangyan Yi, Jianhua Tao +5
Instruct Text-to-Speech (InstructTTS) leverages natural language descriptions as style prompts to guide speech synthesis. However, existing InstructTTS methods mainly rely on a dir…
AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models
Zheng Lian, Haoyu Chen, Lan Chen +9
The emergence of multimodal large language models (MLLMs) advances multimodal emotion recognition (MER) to the next level, from naive discriminative tasks to complex emotion unders…
OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognition
Zheng Lian, Haiyang Sun, Licai Sun +13
Multimodal Emotion Recognition (MER) is a critical research area that seeks to decode human emotions from diverse data modalities. However, existing machine learning methods predom…
AffectGPT: Dataset and Framework for Explainable Multimodal Emotion Recognition
Zheng Lian, Haiyang Sun, Licai Sun +3
Explainable Multimodal Emotion Recognition (EMER) is an emerging task that aims to achieve reliable and accurate emotion recognition. However, due to the high annotation cost, the…
MER 2024: Semi-Supervised Learning, Noise Robustness, and Open-Vocabulary Multimodal Emotion Recognition
Zheng Lian, Haiyang Sun, Licai Sun +15
Multimodal emotion recognition is an important research topic in artificial intelligence. Over the past few decades, researchers have made remarkable progress by increasing the dat…
Multimodal Fusion with Pre-Trained Model Features in Affective Behaviour Analysis In-the-wild
Zhuofan Wen, Fengyu Zhang, Siyuan Zhang +6
Multimodal fusion is a significant method for most multimodal tasks. With the recent surge in the number of large pre-trained models, combining both multimodal fusion methods and p…
Can Deception Detection Go Deeper? Dataset, Evaluation, and Benchmark for Deception Reasoning
Kang Chen, Zheng Lian, Haiyang Sun +4
Deception detection has attracted increasing attention due to its importance in real-world scenarios. Its main goal is to detect deceptive behaviors from multimodal clues such as g…
MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition
Zheng Lian, Licai Sun, Yong Ren +5
Multimodal emotion recognition plays a crucial role in enhancing user experience in human-computer interaction. Over the past few decades, researchers have proposed a series of alg…
SVFAP: Self-supervised Video Facial Affect Perceiver
Licai Sun, Zheng Lian, Kexin Wang +5
Video-based facial affect analysis has recently attracted increasing attention owing to its critical role in human-computer interaction. Previous studies mainly focus on developing…
ITEACH-Net: Inverted Teacher-studEnt seArCH Network for Emotion Recognition in Conversation
Haiyang Sun, Zheng Lian, Chenglong Wang +4
There remain two critical challenges that hinder the development of ERC. Firstly, there is a lack of exploration into mining deeper insights from the data itself for conversational…
GPT-4V with Emotion: A Zero-shot Benchmark for Generalized Emotion Recognition
Zheng Lian, Licai Sun, Haiyang Sun +5
Recently, GPT-4 with Vision (GPT-4V) has demonstrated remarkable visual capabilities across various tasks, but its performance in emotion recognition has not been fully evaluated.…
Explainable Multimodal Emotion Recognition
Zheng Lian, Haiyang Sun, Licai Sun +13
Multimodal emotion recognition is an important research topic in artificial intelligence, whose main goal is to integrate multimodal clues to identify human emotional states. Curre…
MFSN: Multi-perspective Fusion Search Network For Pre-training Knowledge in Speech Emotion Recognition
Haiyang Sun, Fulin Zhang, Yingying Gao +3
Speech Emotion Recognition (SER) is an important research topic in human-computer interaction. Many recent works focus on directly extracting emotional cues through pre-trained kno…
MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learning
Zheng Lian, Haiyang Sun, Licai Sun +15
The first Multimodal Emotion Recognition Challenge (MER 2023) was successfully held at ACM Multimedia. The challenge focuses on system robustness and consists of three distinct tra…
Fully Automated End-to-End Fake Audio Detection
Chenglong Wang, Jiangyan Yi, Jianhua Tao +6
The existing fake audio detection systems often rely on expert experience to design the acoustic features or manually design the hyperparameters of the network structure. However,…
Two-Aspect Information Fusion Model For ABAW4 Multi-task Challenge
Haiyang Sun, Zheng Lian, Bin Liu +3
In this paper, we propose the solution to the Multi-Task Learning (MTL) Challenge of the 4th Affective Behavior Analysis in-the-wild (ABAW) competition. The task of ABAW is to pred…
EmotionNAS: Two-stream Neural Architecture Search for Speech Emotion Recognition
Haiyang Sun, Zheng Lian, Bin Liu +6
Speech emotion recognition (SER) is an important research topic in human-computer interaction. Existing works mainly rely on human expertise to design models. Despite their success…