Self-supervised representations in speech-based depression detection

Abstract

This paper proposes handling training data sparsity in speech-based automatic depression detection (SDD) using foundation models pre-trained with self-supervised learning (SSL). An analysis of SSL representations derived from different layers of pre-trained foundation models is first presented for SDD, which provides insight to suitable indicator for depression detection. Knowledge transfer is then performed from automatic speech recognition (ASR) and emotion recognition to SDD by fine-tuning the foundation models. Results show that the uses of oracle and ASR transcriptions yield similar SDD performance when the hidden representations of the ASR model is incorporated along with the ASR textual information. By integrating representations from multiple foundation models, state-of-the-art SDD results based on real ASR were achieved on the DAIC-WOZ dataset.

References in corpus (7)

• On the Opportunities and Risks of Foundation Models
• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
• BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition
• Topic Modeling Based Multi-modal Depression Detection
• Text-based depression detection on sparse data
• A Step Towards Preserving Speakers' Identity While Detecting Depression Via Speaker Disentanglement
• Climate and Weather: Inspecting Depression Detection via Emotion Recognition

Cited by in corpus (2)

• Parameter Efficient Finetuning for Speech Emotion Recognition and Domain Adaptation
• Transferring speech-generic and depression-specific knowledge for Alzheimer's disease detection