The Influence of Iconicity in Transfer Learning for Sign Language Recognition

Abstract

Most sign language recognition research relies on Transfer Learning (TL) from vision-based datasets such as ImageNet. Some extend this to alternatively available language datasets, often focusing on signs with cross-linguistic similarities. This body of work examines the necessity of these likenesses on effective knowledge transfer by comparing TL performance between iconic signs of two different sign language pairs: Chinese to Arabic and Greek to Flemish. Google Mediapipe was utilised as an input feature extractor, enabling spatial information of these signs to be processed with a Multilayer Perceptron architecture and the temporal information with a Gated Recurrent Unit. Experimental results showed a 7.02% improvement for Arabic and 1.07% for Flemish when conducting iconic TL from Chinese and Greek respectively.

References in corpus (4)

• A Comprehensive Study on Deep Learning-based Methods for Sign Language Recognition
• SignBERT+: Hand-model-aware Self-supervised Pre-training for Sign Language Understanding
• Adapting Multilingual Neural Machine Translation to Unseen Languages
• Cross-Sign Language Transfer Learning Using Domain Adaptation with Multi-scale Temporal Alignment