SMARTe-VR: Student Monitoring and Adaptive Response Technology for e-Learning in Virtual Reality

Abstract

This work introduces SMARTe-VR, a platform for student monitoring in an immersive virtual reality environment designed for online education. SMARTe-VR aims to collect data for adaptive learning, focusing on facial biometrics and learning metadata. The platform allows instructors to create customized learning sessions with video lectures, featuring an interface with an AutoQA system to evaluate understanding, interaction tools (for example, textbook highlighting and lecture tagging), and real-time feedback. Furthermore, we released a dataset that contains 5 research challenges with data from 10 users in VR-based TOEIC sessions. This data set, which spans more than 25 hours, includes facial features, learning metadata, 450 responses, difficulty levels of the questions, concept tags, and understanding labels. Alongside the database, we present preliminary experiments using Item Response Theory models, adapted for understanding detection using facial features. Two architectures were explored: a Temporal Convolutional Network for local features and a Multilayer Perceptron for global features.

arXiv comment10 pages, 3 figures. Published in ACM Intl. Conf. on Multimedia Workshops (ACM MM Workshops 2025, I2M-MM 25). Also presented at IEEE/CVF Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW) and AAAI Workshop on Artificial Intelligence for Education (AI4EDU)

References in corpus (4)

• Human-Centric Multimodal Machine Learning: Recent Advances and Testbed on AI-based Recruitment
• DeepFace-Attention: Multimodal Face Biometrics for Attention Estimation with Application to e-Learning
• mEBAL2 Database and Benchmark: Image-based Multispectral Eyeblink Detection
• A multimodal dataset for understanding the impact of mobile phones on remote online virtual education