MentaLLaMA: Interpretable Mental Health Analysis on Social Media with Large Language Models

Abstract

With the development of web technology, social media texts are becoming a rich source for automatic mental health analysis. As traditional discriminative methods bear the problem of low interpretability, the recent large language models have been explored for interpretable mental health analysis on social media, which aims to provide detailed explanations along with predictions. The results show that ChatGPT can generate approaching-human explanations for its correct classifications. However, LLMs still achieve unsatisfactory classification performance in a zero-shot/few-shot manner. Domain-specific finetuning is an effective solution, but faces 2 challenges: 1) lack of high-quality training data. 2) no open-source LLMs for interpretable mental health analysis were released to lower the finetuning cost. To alleviate these problems, we build the first multi-task and multi-source interpretable mental health instruction (IMHI) dataset on social media, with 105K data samples. The raw social media data are collected from 10 existing sources covering 8 mental health analysis tasks. We use expert-written few-shot prompts and collected labels to prompt ChatGPT and obtain explanations from its responses. To ensure the reliability of the explanations, we perform strict automatic and human evaluations on the correctness, consistency, and quality of generated data. Based on the IMHI dataset and LLaMA2 foundation models, we train MentalLLaMA, the first open-source LLM series for interpretable mental health analysis with instruction-following capability. We also evaluate the performance of MentalLLaMA on the IMHI evaluation benchmark with 10 test sets, where their correctness for making predictions and the quality of explanations are examined. The results show that MentalLLaMA approaches state-of-the-art discriminative methods in correctness and generates high-quality explanations.

Accepted by WWW 2024

References in corpus (4)

• Training language models to follow instructions with human feedback
• FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
• BARTScore: Evaluating Generated Text as Text Generation
• Towards Interpretable Mental Health Analysis with Large Language Models

Cited by in corpus (7)

• Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Data
• Respond to Change with Constancy: Instruction-tuning with LLM for Non-I.I.D. Network Traffic Classification
• Survey and Experiments on Mental Disorder Detection via Social Media: From Large Language Models and RAG to Agents
• Optimizing Large Language Models for ESG Activity Detection in Financial Texts
• Large Language Models in Mental Health: A Systematic Review of Applications, Innovations, and Ethical Challenges
• Assessing Algorithmic Bias in Language-Based Depression Detection: A Comparison of DNN and LLM Approaches
• Forecasting the Buzz: Enriching Hashtag Popularity Prediction with LLM Reasoning