A Graph Isomorphism Network with Weighted Multiple Aggregators for Speech Emotion Recognition

Abstract

Speech emotion recognition (SER) is an essential part of human-computer interaction. In this paper, we propose an SER network based on a Graph Isomorphism Network with Weighted Multiple Aggregators (WMA-GIN), which can effectively handle the problem of information confusion when neighbour nodes' features are aggregated together in GIN structure. Moreover, a Full-Adjacent (FA) layer is adopted for alleviating the over-squashing problem, which is existed in all Graph Neural Network (GNN) structures, including GIN. Furthermore, a multi-phase attention mechanism and multi-loss training strategy are employed to avoid missing the useful emotional information in the stacked WMA-GIN layers. We evaluated the performance of our proposed WMA-GIN on the popular IEMOCAP dataset. The experimental results show that WMA-GIN outperforms other GNN-based methods and is comparable to some advanced non-graph-based methods by achieving 72.48% of weighted accuracy (WA) and 67.72% of unweighted accuracy (UA).

arXiv commentAccepted by Interspeech 2022

References in corpus (6)

• Adam: A Method for Stochastic Optimization
• Neural Message Passing for Quantum Chemistry
• Geometric deep learning: going beyond Euclidean data
• Representation Learning on Graphs with Jumping Knowledge Networks
• Hierarchical Graph Representation Learning with Differentiable Pooling
• Principal Neighbourhood Aggregation for Graph Nets

Cited by in corpus (1)

• Supervised Contrastive Learning with Nearest Neighbor Search for Speech Emotion Recognition