6 papers
EgoEverything: A Benchmark for Human Behavior Inspired Long Context Egocentric Video Understanding in AR Environment
Qiance Tang, Ziqi Wang, Jieyu Lin +3
Long context egocentric video understanding has recently attracted significant research attention, with augmented reality (AR) highlighted as one of its most important application…
Foveated Instance Segmentation
Hongyi Zeng, Wenxuan Liu, Tianhua Xia +3
Instance segmentation is essential for augmented reality and virtual reality (AR/VR) as it enables precise object recognition and interaction, enhancing the integration of virtual…
Unlocking Visual Secrets: Inverting Features with Diffusion Priors for Image Reconstruction
Sai Qian Zhang, Ziyun Li, Chuan Guo +5
Inverting visual representations within deep neural networks (DNNs) presents a challenging and important problem in the field of security and privacy for deep learning. The main go…
GazeGen: Gaze-Driven User Interaction for Visual Content Generation
He-Yen Hsieh, Ziyun Li, Sai Qian Zhang +5
We present GazeGen, a user interaction system that generates visual content (images and videos) for locations indicated by the user's eye gaze. GazeGen allows intuitive manipulatio…
Neural Architecture Search of Hybrid Models for NPU-CIM Heterogeneous AR/VR Devices
Yiwei Zhao, Ziyun Li, Win-San Khwa +10
Low-Latency and Low-Power Edge AI is essential for Virtual Reality and Augmented Reality applications. Recent advances show that hybrid models, combining convolution layers (CNN) a…
Siracusa: A 16 nm Heterogenous RISC-V SoC for Extended Reality with At-MRAM Neural Engine
Arpan Suravi Prasad, Moritz Scherer, Francesco Conti +9
Extended reality (XR) applications are Machine Learning (ML)-intensive, featuring deep neural networks (DNNs) with millions of weights, tightly latency-bound (10-20 ms end-to-end),…