MC-ViViT: Multi-branch Classifier-ViViT to detect Mild Cognitive Impairment in older adults using facial videos

Abstract

Deep machine learning models including Convolutional Neural Networks (CNN) have been successful in the detection of Mild Cognitive Impairment (MCI) using medical images, questionnaires, and videos. This paper proposes a novel Multi-branch Classifier-Video Vision Transformer (MC-ViViT) model to distinguish MCI from those with normal cognition by analyzing facial features. The data comes from the I-CONECT, a behavioral intervention trial aimed at improving cognitive function by providing frequent video chats. MC-ViViT extracts spatiotemporal features of videos in one branch and augments representations by the MC module. The I-CONECT dataset is challenging as the dataset is imbalanced containing Hard-Easy and Positive-Negative samples, which impedes the performance of MC-ViViT. We propose a loss function for Hard-Easy and Positive-Negative Samples (HP Loss) by combining Focal loss and AD-CORRE loss to address the imbalanced problem. Our experimental results on the I-CONECT dataset show the great potential of MC-ViViT in predicting MCI with a high accuracy of 90.63% accuracy on some of the interview videos.

13 pages, 7 tables, 7 figures, 9 equations

References in corpus (3)

• Facial Expression Recognition with Swin Transformer
• Towards Training Stronger Video Vision Transformers for EPIC-KITCHENS-100 Action Recognition
• Video Vision Transformers for Violence Detection

Cited by in corpus (6)

• Detection of Mild Cognitive Impairment Using Facial Features in Video Conversations
• Data Quality Matters: Suicide Intention Detection on Social Media Posts Using RoBERTa-CNN
• A Review of Deep Learning Approaches for Non-Invasive Cognitive Impairment Detection
• Deep Insights into Cognitive Decline: A Survey of Leveraging Non-Intrusive Modalities with Deep Learning Techniques
• Contrastive learning-based video quality assessment-jointed video vision transformer for video recognition
• Fine-Tuning Video Transformers for Word-Level Bangla Sign Language: A Comparative Analysis for Classification Tasks