46 citations · 131 across the 14 of their papers we have counts for
14 papers
InteractGesture: Progressive Chunk Guidance for Continuous Streaming Co-Speech Gesture Control
Ekkasit Pinyoanuntapong, Ajinkya Deogade, Paul Streli +5
Co-speech gesture generation has made significant progress toward realistic full-body motion from speaker audio, yet existing models lack fine-grained spatial controllability of in…
egoEMOTION: Egocentric Vision and Physiological Signals for Emotion and Personality Recognition in Real-World Tasks
Matthias Jammot, Björn Braun, Paul Streli +2
Understanding affect is central to anticipating human behavior, yet current egocentric vision benchmarks largely ignore the person's emotional states that shape their decisions and…
Human Motion Capture from Loose and Sparse Inertial Sensors with Garment-aware Diffusion Models
Andela Ilic, Jiaxi Jiang, Paul Streli +2
Motion capture using sparse inertial sensors has shown great promise due to its portability and lack of occlusion issues compared to camera-based tracking. Existing approaches typi…
Continual Human-in-the-Loop Optimization
Yi-Chi Liao, Paul Streli, Zhipeng Li +2
Optimal input settings vary across users due to differences in motor abilities and personal preferences, which are typically addressed by manual tuning or calibration. Although hum…
EgoSim: An Egocentric Multi-view Simulator and Real Dataset for Body-worn Cameras during Motion and Activity
Dominik Hollidt, Paul Streli, Jiaxi Jiang +4
Research on egocentric tasks in computer vision has mostly focused on head-mounted cameras, such as fisheye cameras or embedded cameras inside immersive headsets. We argue that the…
SympCam: Remote Optical Measurement of Sympathetic Arousal
Björn Braun, Daniel McDuff, Tadas Baltrusaitis +3
Recent work has shown that a person's sympathetic arousal can be estimated from facial videos alone using basic signal processing. This opens up new possibilities in the field of t…
TapType: Ten-finger text entry on everyday surfaces via Bayesian inference
Paul Streli, Jiaxi Jiang, Andreas Fender +3
Despite the advent of touchscreens, typing on physical keyboards remains most efficient for entering text, because users can leverage all fingers across a full-size keyboard for co…
TouchInsight: Uncertainty-aware Rapid Touch and Text Input for Mixed Reality from Egocentric Vision
Paul Streli, Mark Richardson, Fadi Botros +3
While passive surfaces offer numerous benefits for interaction in mixed reality, reliably detecting touch input solely from head-mounted cameras has been a long-standing challenge.…
SituationAdapt: Contextual UI Optimization in Mixed Reality with Situation Awareness via LLM Reasoning
Zhipeng Li, Christoph Gebhardt, Yves Inglin +3
Mixed Reality is increasingly used in mobile settings beyond controlled home and office spaces. This mobility introduces the need for user interface layouts that adapt to varying c…
EgoPressure: A Dataset for Hand Pressure and Pose Estimation in Egocentric Vision
Yiming Zhao, Taein Kwon, Paul Streli +2
Touch contact and pressure are essential for understanding how humans interact with and manipulate objects, insights which can significantly benefit applications in mixed reality a…
EgoPoser: Robust Real-Time Egocentric Pose Estimation from Sparse and Intermittent Observations Everywhere
Jiaxi Jiang, Paul Streli, Manuel Meier +1
Full-body egocentric pose estimation from head and hand poses alone has become an active area of research to power articulate avatar representations on headset-based platforms. How…
BeliefPPG: Uncertainty-aware Heart Rate Estimation from PPG signals via Belief Propagation
Valentin Bieri, Paul Streli, Berken Utku Demirel +1
We present a novel learning-based method that achieves state-of-the-art performance on several heart rate estimation benchmarks extracted from photoplethysmography signals (PPG). W…
HOOV: Hand Out-Of-View Tracking for Proprioceptive Interaction using Inertial Sensing
Paul Streli, Rayan Armani, Yi Fei Cheng +1
Current Virtual Reality systems are designed for interaction under visual control. Using built-in cameras, headsets track the user's hands or hand-held controllers while they are i…
AvatarPoser: Articulated Full-Body Pose Tracking from Sparse Motion Sensing
Jiaxi Jiang, Paul Streli, Huajian Qiu +4
Today's Mixed Reality head-mounted displays track the user's head pose in world space as well as the user's hands for interaction in both Augmented Reality and Virtual Reality scen…