84 citations · 114 across the 20 of their papers we have counts for
20 papers
Revealing Personality Traits: A New Benchmark Dataset for Explainable Personality Recognition on Dialogues
Lei Sun, Jinming Zhao, Qin Jin
Personality recognition aims to identify the personality traits implied in user data such as dialogues and social media posts. Current research predominantly treats personality rec…
ESCoT: Towards Interpretable Emotional Support Dialogue Systems
Tenggan Zhang, Xinjie Zhang, Jinming Zhao +2
Understanding the reason for emotional support response is crucial for establishing connections between users and emotional support dialogue systems. Previous works mostly focus on…
ECR-Chain: Advancing Generative Language Models to Better Emotion-Cause Reasoners through Reasoning Chains
Zhaopei Huang, Jinming Zhao, Qin Jin
Understanding the process of emotion generation is crucial for analyzing the causes behind emotions. Causal Emotion Entailment (CEE), an emotion-understanding task, aims to identif…
Investigating Pre-trained Audio Encoders in the Low-Resource Condition
Hao Yang, Jinming Zhao, Gholamreza Haffari +1
Pre-trained speech encoders have been central to pushing state-of-the-art results across various speech understanding and generation tasks. Nonetheless, the capabilities of these e…
NAIST-SIC-Aligned: an Aligned English-Japanese Simultaneous Interpretation Corpus
Jinming Zhao, Yuka Ko, Kosuke Doi +3
It remains a question that how simultaneous interpretation (SI) data affects simultaneous machine translation (SiMT). Research has been limited due to the lack of a large-scale tra…
MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learning
Zheng Lian, Haiyang Sun, Licai Sun +15
The first Multimodal Emotion Recognition Challenge (MER 2023) was successfully held at ACM Multimedia. The challenge focuses on system robustness and consists of three distinct tra…
Exploiting modality-invariant feature for robust multimodal emotion recognition with missing modalities
Haolin Zuo, Rui Liu, Jinming Zhao +2
Multimodal emotion recognition leverages complementary information across modalities to gain performance. However, we cannot guarantee that the data of all modalities are always pr…
Self-supervised Rewiring of Pre-trained Speech Encoders: Towards Faster Fine-tuning with Less Labels in Speech Processing
Hao Yang, Jinming Zhao, Gholamreza Haffari +1
Pre-trained speech Transformers have facilitated great success across various speech processing tasks. However, fine-tuning these encoders for downstream tasks require sufficiently…
Towards Relation Extraction From Speech
Tongtong Wu, Guitao Wang, Jinming Zhao +4
Relation extraction typically aims to extract semantic relationships between entities from the unstructured text. One of the most essential data sources for relation extraction is…
RedApt: An Adaptor for wav2vec 2 Encoding \\ Faster and Smaller Speech Translation without Quality Compromise
Jinming Zhao, Hao Yang, Gholamreza Haffari +1
Pre-trained speech Transformers in speech translation (ST) have facilitated state-of-the-art (SotA) results; yet, using such encoders is computationally expensive. To improve this,…
Generating Synthetic Speech from SpokenVocab for Speech Translation
Jinming Zhao, Gholamreza Haffar, Ehsan Shareghi
Training end-to-end speech translation (ST) systems requires sufficiently large-scale data, which is unavailable for most language pairs and domains. One practical solution to the…
Multi-Task Learning Framework for Emotion Recognition in-the-wild
Tenggan Zhang, Chuanhe Liu, Xiaolong Liu +7
This paper presents our system for the Multi-Task Learning (MTL) Challenge in the 4th Affective Behavior Analysis in-the-wild (ABAW) competition. We explore the research problems o…
M-Adapter: Modality Adaptation for End-to-End Speech-to-Text Translation
Jinming Zhao, Hao Yang, Ehsan Shareghi +1
End-to-end speech-to-text translation models are often initialized with pre-trained speech encoder and pre-trained text decoder. This leads to a significant training gap between pr…
M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Database
Jinming Zhao, Tenggan Zhang, Jingwen Hu +4
The emotional state of a speaker can be influenced by many different factors in dialogues, such as dialogue scene, dialogue topic, and interlocutor stimulus. The currently availabl…
Multi-modal Emotion Estimation for in-the-wild Videos
Liyu Meng, Yuchen Liu, Xiaolong Liu +5
In this paper, we briefly introduce our submission to the Valence-Arousal Estimation Challenge of the 3rd Affective Behavior Analysis in-the-wild (ABAW) competition. Our method uti…
MEmoBERT: Pre-training Model with Prompt-based Learning for Multimodal Emotion Recognition
Jinming Zhao, Ruichen Li, Qin Jin +2
Multimodal emotion recognition study is hindered by the lack of labelled corpora in terms of scale and diversity, due to the high annotation cost and label ambiguity. In this paper…
It is Not as Good as You Think! Evaluating Simultaneous Machine Translation on Interpretation Data
Jinming Zhao, Philip Arthur, Gholamreza Haffari +2
Most existing simultaneous machine translation (SiMT) systems are trained and evaluated on offline translation corpora. We argue that SiMT systems should be trained and tested on r…
MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation
Jingwen Hu, Yuchen Liu, Jinming Zhao +1
Emotion recognition in conversation (ERC) is a crucial component in affective dialogue systems, which helps the system understand users' emotions and generate empathetic responses.…
WenLan: Bridging Vision and Language by Large-Scale Multi-Modal Pre-Training
Yuqi Huo, Manli Zhang, Guangzhen Liu +32
Multi-modal pre-training models have been intensively explored to bridge vision and language in recent years. However, most of them explicitly model the cross-modal interaction bet…
MUSE2020 challenge report
Ruichen Li, JingWen Hu, Shuai Guo +1
This paper is a brief report for MUSE2020 challenge. We present our solution for Muse-Wild sub challenge. The aim of this challenge is to investigate sentiment analysis method in r…