Depression detection in social media posts using transformer-based models and auxiliary features

Abstract

The detection of depression in social media posts is crucial due to the increasing prevalence of mental health issues. Traditional machine learning algorithms often fail to capture intricate textual patterns, limiting their effectiveness in identifying depression. Existing studies have explored various approaches to this problem but often fall short in terms of accuracy and robustness. To address these limitations, this research proposes a neural network architecture leveraging transformer-based models combined with metadata and linguistic markers. The study employs DistilBERT, extracting information from the last four layers of the transformer, applying learned weights, and averaging them to create a rich representation of the input text. This representation, augmented by metadata and linguistic markers, enhances the model's comprehension of each post. Dropout layers prevent overfitting, and a Multilayer Perceptron (MLP) is used for final classification. Data augmentation techniques, inspired by the Easy Data Augmentation (EDA) methods, are also employed to improve model performance. Using BERT, random insertion and substitution of phrases generate additional training data, focusing on balancing the dataset by augmenting underrepresented classes. The proposed model achieves weighted Precision, Recall, and F1-scores of 84.26%, 84.18%, and 84.15%, respectively. The augmentation techniques significantly enhance model performance, increasing the weighted F1-score from 72.59% to 84.15%.

Social Network Analysis and Mining (Accepted)

References in corpus (11)

• Utilizing Neural Networks and Linguistic Metadata for Early Detection of Depression Indications in Text Sequences
• Calibration of Transformer-based Models for Identifying Stress and Depression in Social Media
• DEPTWEET: A Typology for Social Media Texts to Detect Depression Severities
• Explainable Identification of Dementia from Transcripts using Transformer Networks
• Utilizing BERT Intermediate Layers for Aspect Based Sentiment Analysis and Natural Language Inference
• Detecting Dementia from Speech and Transcripts using Transformers
• A Multimodal Framework for Depression Detection during Covid-19 via Harvesting Social Media: A Novel Dataset and Method
• Mental Illness Classification on Social Media Texts using Deep Learning and Transfer Learning
• Multitask learning for recognizing stress and depression in social media
• A Multimodal Approach for Dementia Detection from Spontaneous Speech with Tensor Fusion Layer
• Explainable Verbal Deception Detection using Transformers

Cited by in corpus (1)

• A Graph Neural Architecture Search Approach for Identifying Bots in Social Media