Multimodality in VR: A survey
arXiv:2101.07906 · doi:10.1145/3508361
Abstract
Virtual reality (VR) is rapidly growing, with the potential to change the way we create and consume content. In VR, users integrate multimodal sensory information they receive, to create a unified perception of the virtual world. In this survey, we review the body of work addressing multimodality in VR, and its role and benefits in user experience, together with different applications that leverage multimodality in many disciplines. These works thus encompass several fields of research, and demonstrate that multimodality plays a fundamental role in VR; enhancing the experience, improving overall performance, and yielding unprecedented abilities in skill and knowledge transfer.
35 pages (24 pages not including references), 10 figures, 4 tables
References in corpus (9)
- Towards Low-Latency and Ultra-Reliable Virtual Reality
- Predicting Head Movement in Panoramic Video: A Deep Reinforcement Learning Approach
- Movie Editing and Cognitive Event Segmentation in Virtual Reality Video
- Self-Supervised Generation of Spatial Audio for 360 Video
- ScanGAN360: A Generative Model of Realistic Scanpaths for 360 Images
- Impact of Information Placement and User Representations in VR on Performance and Embodiment
- How do people explore virtual environments?
- VisualEchoes: Spatial Image Representation Learning through Echolocation
- Virtual Environments for Rehabilitation of Postural Control Dysfunction