$M Recognizer for User-Defined Mid-Air Gestures

Abstract

User-defined mid-air gestures hold promise to customize device interactions to a user's preferences, cultural background, and physical abilities. This is especially true for individuals with upper-body motor disabilities, who may find standardized mid-air gestures inaccessible. However, the biosignals that enable such mid-air gesture recognition are multi-modal, multi-dimensional, and noisy, requiring domain expertise to design input features for gesture recognizers. The $M-recognizer enables mid-air gesture recognition without needing expertise in biosignals or algorithms. $M resamples, normalizes, and performs dimensionality reduction to reduce noise and enhance signals relevant to the recognition. We evaluated $M on 26 participants with and 8 participants without upper-body motor disabilities performing user-defined ability-based gestures. For user-dependent and gesture articulation variability conditions, $M outperformed our comparison algorithms (traditional machine learning with expert features and deep learning), with >95% recognition rate. We tested $M's ease-of-implementation with six coders with minimal machine learning experience. Participants implemented $M 53% faster and with 23% lower workload compared to deep learning. $M highlights the potential and feasibility of user-defined and accessible mid-air gesture interfaces.

arXiv comment29 pages, 10 figures, 1 table